Hindawi Publishing Corporation
International Journal of Digital Multimedia Broadcasting
Volume 2010, Article ID 352060, 12 pages
doi:10.1155/2010/352060

Research Article
ipProjector: Designs and Techniques for Geometry-Based
Interactive Applications Using a Portable Projector
Thitirat Siriborvornratanakul and Masanori Sugimoto
Interaction Technology Laboratory, Department of Electrical Engineering and Information Systems, The University of Tokyo,
7-3-1 Hongo, Bunkyo-ku, Tokyo 113-8656, Japan
Correspondence should be addressed to Thitirat Siriborvornratanakul, thitirat@itl.t.u-tokyo.ac.jp
Received 1 September 2009; Revised 23 November 2009; Accepted 14 December 2009
Academic Editor: Jungong Han
Copyright © 2010 T. Siriborvornratanakul and M. Sugimoto. This is an open access article distributed under the Creative
Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided theoriginal work is properly cited.
We propose an interactive projection system for a virtual studio setup using a single self-contained and portable projection device.
The system is named ipProjector, which stands for Interactive Portable Projector. Projection allows special eﬀects of a virtual studio
to be seen by live audiences in real time. The portable device supports 360-degree shooting and projecting angles and is easy to
be integrated with an existing studio setup. We focus on two fundamental requirements of the system and their implementations.
First, nonintrusive projection is performed to ensure that the special eﬀect projections and the environment analysis (for locating
the target actors or objects) can be performed simultaneously in real time. Our approach uses Digital Light Processing technology,
color wheel analysis, and nearest-neighbor search algorithm. Second, a paired projector-camera system is geometrically calibrated
with two alternative setups. The first uses a motion sensor for real-time geometric calibration, and the second uses a beam splitterfor scene-independent geometric calibration. Based on a small-scale laboratory setting, experiments were conducted to evaluate
the geometric accuracy of the proposed approaches, and an application was built to demonstrate the proposed ipProjector concept.
Techniques of special eﬀect rendering are not concerned in this paper.

1. Introduction
Recently, virtual studio setups have become popular for
modern studio productions. Techniques such as studio camera tracking and 3D graphic rendering have been integrated
into a conventional studio setup, so professional special
eﬀects can be created at lower cost. The main problem withcurrent virtual studio setups is that these special eﬀects areinvisible during film recording. Therefore, the special eﬀects
cannot be shown to live audiences in live broadcasting. In
addition to actors and moderators, it is diﬃcult to respondcorrectly to invisible content.
One solution based on existing technologies uses a
projector. By combining a projector and a camera, any
visible or invisible special eﬀect can be directly superimposed
onto a studio surface. The visible special eﬀects support
live broadcasting, provide realism during virtual studiorecording, and allow direct interaction with actors. The
invisible special eﬀect, which can only be seen by a specificcamera, works very well as a supporting system for virtual
studio production. Moderators can embed hidden scripts forverification. In an advanced virtual studio system like that
proposed in [1], the hidden information is used to accurately
track studio cameras and to render real-time 3D graphics inan arbitrary studio environment.
Using a projector and a camera for live broadcasting withspecial eﬀects is one type of real-time interactive projection
application. By using a projector-camera paired system (apro-cam system), we propose a system that includes hardware
designs and software implementations. The system is selfcontained, portable, and performs real-time projection andreal-time environment analysis simultaneously.
The self-contained portable device allows easy setups
in any arbitrary studio environment. Furthermore, theshooting and projecting angles of the camera and projector
can be determined freely based on the director’s consideration. This is important in actual film shooting where thehighest priority is to get the best images from the desirable

2
angle shot. The simultaneous real-time projection and realtime environment analysis allows advanced virtual studiotechniques to be performed while special eﬀects are being
continuously projected. Environment analysis in this context
refers to the analysis of three targets: the studio scenery andobjects, and inside actors. For example, captured images areanalyzed to locate an inside actor and his postures so thatspecial eﬀects can be created and projected in response to
his actions. Note that the concept proposed in this paper
could also be applied to other portable interactive projection
systems.
This paper focuses on two problems that are important in
implementing real-time interactive projection applications.
The first problem is projecting real-time special eﬀects sothat they do not interfere with normal camera capturing.
The second problem is calibrating a pro-cam system precisely
to ensure correct transformations between a projector and acamera in a portable setup, which implies use in an unknown
environment.
The first problem is also called the nonintrusive projection problem. In normal projection, projected images canbe captured by a camera; discriminating between projected
contents and real contents in captured images is still diﬃcult.
Nonintrusive projection guarantees that projected special
eﬀects will be visible to humans and normal studio cameras
but invisible to the calibrated camera. This is necessary to
avoid projective interference (as seen by the camera) that may
lead to an incorrect environment analysis. The techniqueproposed in this paper involves Digital Light Processing(DLP) projector, RGB color space sampling and nearestneighbor search algorithm.
With respect to the second problem, there are two related
calibrations that are often concerned by researchers using
a pro-cam system: geometric and radiometric calibrations.
Both calibrations require completely diﬀerent calibration
approaches. In this paper, we focus on the first calibrationto achieve a geometrically calibrated pro-cam system. With
the calibrated system, we are able to project special eﬀects
onto desired locations. A motion-sensor-based calibration
technique and a beam-splitter-based calibration techniqueare investigated in the following sections.
In this paper, we show two complete designs of the selfcontained portable projection device. Based on the developed designs, solutions of the nonintrusive projection problem and the geometric calibration problem are proposed.
Experiments were conducted to evaluate the accuracy ofthe proposed approaches in a small-scale laboratory setting;a white board and color magnets were used to represent
the studio projection surface and the studio target objects,
respectively. Techniques of special eﬀect rendering are notconcerned in this paper; therefore, the projected special
eﬀects are the simple “+” pattern. The proposed devices
and approaches can be applied in a full-scale virtual studioenvironment by implementing additional target detection
algorithms for the desired studio target objects or actors.
The rest of this paper is laid out as follows. Section 2
explains recent advances in interactive projector systems andthen discusses related research in the two problem areas asmentioned earlier. Section 3 shows the techniques using an

International Journal of Digital Multimedia Broadcastingoﬀ-the-shelf DLP projector for nonintrusive projection; theinitial setup for pro-cam synchronization, necessary camera
settings and detailed analysis results regarding our DLP
projector model are written in the section. Section 4 presents
two alternative system configurations (in combination with
the initial setup shown in Section 3) for real-time pro-cam
geometric calibration based on perspective transformation
model. On one hand, Section 4.1 applies an additionalmotion sensor for calibration on a planar or slanted surface.
On the other hand, Section 4.2 uses a beam splitter and introduces a complete portable design consisting of a projector,
camera and beam splitter; the precise geometric calibrationis achieved on both planar and nonplanar surface and suitability of the design regarding the nonintrusive projection
is observed in the section. In Section 5, performances of the
proposed nonintrusive projection and geometric calibration
approaches are experimentally confirmed and an applicationis built to demonstrate the overall concept of the ipProjector.
Finally, Section 6 concludes this article along with a plan for
future works.

2. Related Work2.1. Interactive Projector Systems. Cotting and Gross [2]introduced an environment-aware display system that automatically avoids projections onto nonsurface objects. Theirsystem performs real-time interactions, but is limited to
fixed projectors and fixed cameras mounted on a ceiling. In
addition, surfaces are restricted to flat table surfaces whose
distance to the ceiling is unchanged. In Cao et al. [3, 4],interactive mobile projector systems were developed. While
their projection device is self-contained, it requires a camera
mounted separately in a workspace for 3D positioning
purpose. In CoGAME [5], images projected from a handheld
projector control the movement of a robot. A camera withan IR filter sees only three IR LEDs attached to the robot,
and so other visual information about the environment
is disregarded completely. The latest SixthSense prototype[6] is a mobile pro-cam device that oﬀers meaningfulinteractions with diﬀerent objects found in the real world.
However, the projector and camera are not calibrated in theirsystem and environment analysis is performed even though
there is projective interference. Consequently, geometricaccuracy is limited and color markers are used to help
locate target objects like fingertips and desired projection
areas. Unlike these systems, our proposed interactive system
is truly self-contained and geometrically calibrated. It canperform real-time projection and real-time environment
analysis simultaneously without any projective interference.
2.2. Nonintrusive Projection. This topic is a subset of theembedded imperceptible-pattern projection problem. Prototypes of an infrared projector were proposed in [7, 8]
to project infrared and visible light simultaneously. An
infrared pattern is fixed by using an internal mask insidea projector in [7], but is variable in [8]. Unfortunately,
the work of Lee et al. [8] requires many internal changes
inside a DLP projector that can be accomplished only byInternational Journal of Digital Multimedia Broadcasting
a commercial manufacturer. While an infrared projector isunder investigation, there are existing solutions proposed
for this problem. For the oﬃce of the future [9], structured
light can be embedded into a DLP projector by making
significant changes to the projection hardware. However, this
implementation is impossible unless it is incorporated into
the design of the projector or full access to the projection
hardware is available. In [1, 10, 11], a code image is projected
at high speed with its neutralized image, which integrates
the coded patterns invisibly due to limitations of thehuman visual system. According to these papers, projecting
and capturing at 120 Hz can guarantee a hidden code.
Commonly available projectors usually perform projections
at a maximum rate of 87 Hz.
For this paper, we apply an approach based on the DLP
characteristics. Using the camera classification approach
proposed in [12] and the nearest-neighbor search algorithm,
we are able to perform nonintrusive projection using an oﬀthe-shelf DLP projector.
2.3. Real-Time Pro-Cam Geometric Calibration. When aprojector and a camera are rigidly fixed to each other,some have assumed that the geometric registration between
them is roughly constant [13]. However, as the angleof the projector moves from the perpendicular or as asurface becomes nonplanar, this approach will no longer
guarantee good geometric registration. Projecting a known
pattern onto a surface is a classical approach to solve
this problem that gives precise calibrations for both planar
surfaces [14–16] and irregular surfaces [17–19]. However,the computational cost is high for complex surfaces, andpatterns must be re-projected when a component of thesystem (e.g., a projector, camera or surface) moves. Similar
approach is applied in the catadioptric projectors [20] whose
projected light is refracted/reflected by refractors/reflectors;
geometric registration between the two devices is obtainedby projecting a series of known patterns and allowing thecamera to sense them. A real-time approach that does
not interrupt normal projection was proposed in [21] byattaching four laser-pens to a pro-cam system. Althoughdetecting bright laser points sounds easier than detecting
points projected by a projector, locating small laser points ina messy camera image is still diﬃcult. In [22], Johnson andFuchs proposed a real-time approach that does not interrupt
normal projection, requires no fixed marker and can beapplied to a complex surface. By matching feature points
found in the projected image and the predicted captured
image, the pose of the projector is tracked and the calibrationis achieved in real time. However, the camera is stationary
and separated from the projector in their system.
This paper involves two calibration approaches that canbe implemented as a single self-contained device. The first
approach uses one additional motion sensor, and geometriccalibration is achieved in real time on a planar or slanted
surface. The second approach uses a beam splitter tocoaxialize the projector and camera. Geometric calibration
is independent of the scene, so both planar and irregularsurfaces can be used as projection surfaces.

3

3. Nonintrusive Projection
As mentioned in the introduction, it is important that areal-time environment analysis has no interference from any
projected contents. If it does, the system might consider the
projected content as a real target and generate special eﬀects
in response to that false detection.
Recently, internal characteristics of a DLP projector havereceived lot of attentions from research communities. On one
hand, in [23, 24], the dithered illumination pattern corresponding to the DMD chip (which operates at 10 000 Hz)
is observed and utilized using a very high speed camera(whose maximum speed is 3000 fps). On the other hand,characteristics of the color wheels (which rotate at 120 Hz)
can be investigated and used by a camera with slower
capturing speed. Our nonintrusive projection is based on
the latter one; characteristics of the color wheels are utilized
here for nonintrusive projection purpose. The proposed
approach has three main advantages: it requires no internalchange to the projector or the camera, it can be applied toany oﬀ-the-shelf DLP projector, and it supports embedded
variable light patterns in the future without further hardware
modifications.
In the following sections, we explain in detail how toanalyze the characteristics of the color wheels inside the DLP
projector and how to use these characteristics for nonintrusive projection. Note that a beam splitter (as described in
Section 4.2) has not yet been applied in these sections.
3.1. DLP Projector Analysis. Because each DLP projector
model owns unique characteristics of the inside color
wheels, DLP projector analysis has to be performed before
using an unknown DLP projector model for the proposed
nonintrusive projection approach. To understand the overallcharacteristics of the color wheels without full access to a
DLP chip and its controller, we applied the camera-basedclassification method proposed in [12]. In this section,
we briefly explain the classification steps and show theclassification results of our DLP projector.
First, this classification method requires a camera withan external trigger feature to synchronize it with a DLP
projector. Synchronization between the projector and thecamera is performed by tapping the vertical sync signal (5 V,
60 Hz) from the computer to the projector. By using thetapped signal as a trigger, our camera remains synchronized
to the projector. In addition, the shutter of the camera must
be set to open for a very short period in order to sense the fastcharacteristics of the color wheels. In our setup, the camera
is set to expose for only 0.55 ms.
The following devices were used for our pro-cam synchronization: a HP MP2225 DLP projector (XGA 1024 ×
768 projection resolution) with a D-sub 15 pin connector,a Dragonfly Express camera (VGA 640 × 480 captured
resolution) connected through a FireWire 800 (IEEE1394B)
port, and an ELECOM VSP-A2 VGA splitter. The camera isequipped with a Tamron 13VM308AS lens. The synchronization setup is illustrated in Figure 1.
Second, we analyzed the overall sequences of the color
wheels inside our DLP projector by projecting single-color

images (corresponding to the colors of each available color
wheel of the projector) at maximum intensity through all
possible starting exposure times. Figure 2 was created byallowing the synchronized camera to sense these projectedcolors with diﬀerent starting exposure times.
Third, we analyzed detailed mirror flip sequences forall 256 values in the selected color channel (i.e., red, green
or blue) within a narrow starting exposure period. From
Figure 2, the selected color channel is the red channel, whichis the first channel appearing in the sequences. Mirror flip
sequences were then obtained by projecting uniform red
images with intensity values ranging from 0 to 255, andwith the starting exposure times ranging from 0 to 2 ms.
Figure 3 was created by allowing the camera to sense these
red projections. A starting exposure time of 1.4 ms was finallychosen because it provides the best distributed red ramps, as
shown in Figure 3.
Following the explained steps, we are able to synchronize
the camera with the projector at the appropriate starting
exposure time. For our selected starting exposure time, thecamera can only see the red light of the DLP projector(correspondences between projected red intensities and red
intensities seen by the camera are shown in Figure 3). If
red intensities of the projected special eﬀects are similar to
those of the projected background color (as seen by thecamera), the system cannot diﬀerentiate the projected special
eﬀects from the background in captured images. Hence,
further environment analysis is not interfered by the realtime special eﬀect projection.

International Journal of Digital Multimedia BroadcastingComputer

VGA splitter
VGA

Image

VGA

VGA

V-syncGroundFirewirecameraDLP projector

Figure 1: Setup for projector-camera synchronization.

HP-MP2225 projector

4

0

1

2

3

4

5
(ms)

6

7

8

9

10

Figure 2: Color-wheel sequences of the HP MP2225 as seen by thesynchronized camera with starting exposure times ranging from 0
to 10 ms.

255

3.2. Environment Illumination. To perform the DLP analysis
and use it for nonintrusive projection (as mentioned in
Section 3.1), the shutter of the camera is set to expose for
only 0.55 ms, which is too short for the camera to sense theenvironment properly (as shown in Figure 4(b)) unless there
is light emitting from the projector (as shown in Figure 4(c)).
In [12], the 256 red intensities in the selected timeslot wereclassified into three sets: white, black and grey. White refers tocolors whose projection fully turns mirrors inside the DLP
projector and transmits lights toward a surface. Black refers
to colors whose projection does not flip the mirrors andtransmits no light toward a surface. Grey refers to unreliablestates between white and black.
For environment analysis purpose, we need to illuminate the environment while projecting nonintrusive special
eﬀects. Thus, only colors whose red value contained in thewhite set should be projected. Figure 4(c) depicts an image
seen by the camera when Figure 4(d), whose red intensity is
maximal for all pixels, was projected. Note that the intensityof Figure 4(b) was enhanced here to allow the environment
to be seen.
3.3. Color Space Sampling and Color Conversion. The methodof projecting only colors whose red intensity contained in thewhite set (as mentioned in Section 3.2) works only for a DLP
projector model that does not have interdependent color
channels in mirror flipping. Instead of depending on the redchannel and risking eﬀects from the other color channels,

191
128
64
0
0

0.5

1

1.5

2

1.4 ms

(ms)
(a)

(b)

Figure 3: (a) Detailed mirror flip sequences for all 256 values in the
red channel with the starting exposure times ranging from 0 to 2 ms.
(b) Mirror flip sequences at the selected starting exposure time.

we propose a DLP-model-independent classification thatinvolves all color channels (i.e., red, green and blue) ofprojected images. Each color in the RGB color space wasprojected onto the white surface and sensed by the camera.
Using the camera classification method, these projectedcolors were classified into the three sets previously discussed.
Based on this proposed classification, we are also able todetermine whether our DLP projector has interdependentcolor channels in mirror flipping.
From the entire RGB color space containing 256 × 256
× 256 colors, we selected only 11 × 11 × 11 colors andobtained 115 colors belonging to the white set. Because theamount of light passing through the camera is decreased by

International Journal of Digital Multimedia Broadcasting5
BeforeAfter (white)

(a)
(a)

(b)

(c)
After (black)

(b)

(d)

(c)

(d)

Figure 4: (a) is an environment seen by a camera with normal shutter settings. (b) is the environment (a) as seen by the synchronizedcamera. (c) is the environment (a) seen by the synchronized camera
when image (d) is being projected from the DLP projector.

the beam splitter setup (as detailed in Section 4.2), we set ahigh threshold value for the white set. Thus, there are few
colors categorized into this set. In detail, we projected singlecolor images (corresponding to the colors in the selectedcolor space) and allowed the camera to sense the projectedcolors. For one projected color, values of all pixels inside theprojection area (as seen by the camera) are averaged before
that color is classified into the appropriate set. Because ofthis, we can ensure that the diﬀerent resolutions between the
projector and camera will not aﬀect our processes of color
space sampling and color classification.
For converting an arbitrary color into the most similarcolor in the white set, we used an approximate nearestneighbor search algorithm called Best Bin First (BBF) [25].
Using this, we can convert an arbitrary color into the whiteset, and the working environment is then always illuminated.
In Figure 5, we randomly generated 400 colors and converted
them into the white set using the BBF algorithm. The
converted image was then projected and sensed by thesynchronized camera. Conversion of the same image tothe black set is also shown for comparison. From images
captured by the camera shown in Figure 5, it is clear that
our classification and conversion approach can perform
eﬃciently, and the surface remains well illuminated when
projecting colors in the white set.

4. Real-Time Pro-Cam Geometric Calibration
Geometric calibration is the first problem encountered bymost pro-cam systems. Geometric mapping between cameraand projector coordinates is necessary to find corresponding
positions between the two coordinate systems, and to project
images back to desired locations on an actual surface. The
three objects aﬀecting this calibration are the projector, thecamera and the surface. Any relative movement between(e)Figure 5: Color conversion using BBF algorithm. (a) is therandomly generated 400 colors before conversion. (b) and (d) arethe (a) image after converted to the white and black set, respectively.
(c) and (e) are the blank surface seen by the synchronized camera
when image (b) and (d) are being projected, respectively.

any pair of the three objects causes changes in the procam geometric mapping. By attaching the camera rigidly
to the projector, there is no relative movement between the
projector and the camera but this cannot prevent relative
movements with the surface. On a planar surface, when
the angle of a projector moves away from perpendicular,
the geometric mapping changes. Nonoverlapping fields ofview of the projector and the camera may make geometricmapping impossible in some positions. On an irregularsurface, there is an additional serious problem as 3D shapesof the surface create parallax eﬀects between projector andcamera coordinates. This problem is diﬃcult to recover fromunless the 3D geometries of the surface are known.
In this paper, geometric mapping between camera andprojector coordinates is computed by perspective transformation [26] whose computation eﬀort is lighter than
Euclidean calculation does. Based on the fact that all points
seen by the camera lay on some unknown plane, theperspective transformation between the two coordinates canbe established by a 3 × 3 homography matrix. Suppose that
(X, Y ) is a pixel in projector coordinates whose corresponding pixel in camera coordinates is (x, y), the perspective
transformation from (x, y) to (X, Y ) can be expressed witheight degrees of freedom in(X, Y ) =

h 1 x + h2 y + h3 h 4 x + h5 y + h6,,
h 7 x + h8 y + h9 h 7 x + h8 y + h9

(1)
−
→

−
→

where h = (h1 · · · h9 )T is constrained by condition | h | = 1.
The same transformation as written in (1) can be expressedin homogeneous coordinates as
⎛

Xw

⎞

⎛

h1 h2 h3

⎞ ⎛ ⎞

x

⎜
⎟ ⎜
⎟ ⎜ ⎟
⎜Y w ⎟ = ⎜h4 h5 h6 ⎟ · ⎜ y ⎟.
⎝
⎠ ⎝
⎠ ⎝ ⎠

w
−
→

h7 h8 h9(2)1

h can be computed from four corresponding pixels between
the two coordinates (four correspondences ensure that no6International Journal of Digital Multimedia Broadcasting

three points is collinear). When there are more than four
corresponding pixels found between the two coordinates(δ > 4 in (3)), the RANSAC method is applied for estimating−→
the values of h in the following equation:

Motion sensor

VGA splitter

⎞ ⎛
⎞ ⎛
⎞
X1 w X2 w · · · Xδ w
h1 h2 h3x1 x2 · · · xδ
⎜
⎟ ⎜
⎟ ⎜
⎟
⎜Y1 w Y2 w · · · Yδ w ⎟=⎜h4 h5 h6 ⎟ · ⎜ y1 y2 · · · yδ ⎟.
⎝
⎠ ⎝
⎠ ⎝
⎠
w
w ··· wh7 h8 h9
1 1 ··· 1⎛(3)The purpose of the following sections (i.e., Sections 4.1 and
4.2) is to find at least four corresponding pixels between
camera and projector coordinates and use them to compute−→the updated h values in real time. After that, geometricmapping from any (x, y) to (X, Y ) and vice versa is achieved−→

Firewire camera
(a)

−
→ −1

using h and [ h ] , respectively. Sections 4.1 and 4.2
explain two alternative approaches for finding a set ofcorresponding pixels between the two coordinates basedon two diﬀerent setups. Both approaches are implemented
as a single self-contained device and support portable use.
The first approach uses an additional motion sensor andcan perform real-time geometric calibration on a planar or
slanted surface. The second approach uses a beam splitter sothat the geometric mapping is independent of the surface.
By the way, considering the frame buﬀer architecture of
the projector, there is one frame delay time before an image
sent to the projector will be projected out. Any movement
regarding the projector, camera or surface during this
delay time causes geometric errors between the calculated
geometries and the actual geometries (of projected images)
appearing on the surface. In our system, frequency of the
projection cycle is 60 Hz (as mentioned in Section 3.1)
which equals to the maximum 16.67 ms projection delay
time. For the application proposed in Section 5, this delay
time is relatively short compared to other processing times.
Therefore, we decided to neglect the eﬀect of this delay
from our calculation. However, for an interactive system that
requires the millisecond precision in projection, additional
techniques such as motion estimation should be applied.
4.1. Motion-Sensor-Based Approach. This section describes
the approach using a motion sensor to find a set of corresponding pixels between camera and projector coordinates
on a planar or slanted surface. Two tilt sensors fixed toa projector were first proposed in [16]. Acquiring the tilt
angles from both sensors in real time allows the correct
estimation of the world’s horizontal and vertical directions
without using markers. Dao et al. [27] extended the sensorbased idea to an accelerometer combined with a digitalcompass. Their system measures the inclined angle of a
projector directly in both vertical and horizontal axes, andthen creates an interactive application by using real-time
keystone correction.
A sensor eliminates the need for fiducial markers but stillallows a single self-contained device. Our configuration forthis calibration approach is shown in Figure 6; a projector,
camera and motion sensor are fixed together on a wooden

DLP projector

Computer

IEEE
1394B

USB
VGA

FirewirecameraDLP projector
MotionsensorSynchronized(b)

Figure 6: Projector-camera-sensor configuration.

base so that their relative positions and orientations cannot
be changed. The purpose is to obtain pairs of corresponding(X, Y ) and (x, y) in real time. Unlike previous researches
that use the sensors to compute the rotation matrix of a
projector or to correct the keystone distortion, we directly
correlate the sensor values with camera coordinates in order
to find updated geometric mapping between camera andprojector coordinates in real time. Keystone correction is notconcerned in our system.
In this paper, a NEC/TOKIN MDP-A3U9S 3D motion
sensor is used with a data update rate of 125 Hz. The relative
pitch and roll angles are calculated from three acceleration
values, AccX, AccY and AccZ (acceleration values along X,
Y , and Z axis, resp.), read from the accelerometer embedded
in our motion sensor. Setting the reference angles is simple—
the projector is moved until the images appear rectangular ona surface and then a key or button is pressed. The reference
can be reset whenever a user prefers or feels significant
geometric errors (between the calculated geometries andthe actual geometries appearing on the surface) in thecalibration. Five consecutive samples of the 3D acceleration

International Journal of Digital Multimedia Broadcasting

7

values acquired from the sensor are averaged in real time
before being used to compute the relative pitch and rollangles. Averaging adds a delay but is recommended for asmoother calibration. The calculation of the relative pitch
and roll angles can be summarized as follows:⎛

pitchrel = arccos⎝⎞

AccXavg
AccXavg 2 + AccYavg 2 + AccZavg 2⎠(4)− pitchref ,

rollrel = arctan

AccYavg
AccZavg− rollref ,where pitchref and rollref refer to the reference pitch androll angles. AccXavg , AccYavg and AccZavg are the average offive consecutive AccX, AccY and AccZ values read from thesensor, respectively.
This approach requires oﬄine calibration. However, thecalibration data are compatible with the system if there is
no change in the relative positions or orientations of thethree devices. Suppose that the oﬄine calibration is achieved
using N sample images captured from the camera, and all
sample images share a set of n points to be calibrated. A
set of calibration data provided by one sample image can be
written as (p, r, x1 , y1 , x2 , y2 , . . . , xn , yn ). Let p and r refer tothe relative pitch and roll angles, and (xi , yi ) represents the
2D camera coordinate of an ith observed point in the sample
image. For N sample images captured from diﬀerent angles
and orientations, we have⎡⎤p(1) r(1) 1⎢⎢p⎢ (2) r(2)
⎢
A=⎢
..
⎢ ..
⎢ .
.
⎣

⎥

1⎥
⎥
⎥

,
.. ⎥
⎥
.⎥
⎦

p(N) r(N) 1⎡

x1(1) y1(1) x2(1) y2(1) · · · xn(1) yn(1)⎢⎢x⎢ 1(2)
⎢
B=⎢
⎢ ..
⎢ .
⎣

⎤

(5)

⎥

y1(2) x2(2) y2(2) · · · xn(2) yn(2) ⎥
⎥
..
.

..
.

..
.

..

.

..
.

..
.

⎥
⎥.
⎥
⎥
⎦

x1(N) y1(N) x2(N) y2(N) · · · xn(N) yn(N)The oﬄine calibration is finished when an adjustment matrix(β) is obtained by using linear least squares to solve

AT A β = AT B.

(6)

At any time τ during the online calibration, (xi , yi ) is
updated from the relative pitch and roll angles (calculated as
explained earlier) by








x1(τ) , y1(τ) , x2(τ) , y2(τ) , . . . , xn(τ) , yn(τ) = p(τ) , r(τ) , 1 β.
(7)

Following the above explanation, even though the position and orientation of the pro-cam system (relative to thesurface) are not known, the system is able to obtain camera

coordinates (xi(τ) , yi(τ) ) of the n pre-defined points (whose
projector coordinates (Xi(τ) , Yi(τ) ) are known). Using the−→corresponding (xi(τ) , yi(τ) ) and (Xi(τ) , Yi(τ) ) to compute h as
shown in (3) (when δ = n), real-time geometric mapping
between camera and projector coordinates is achieved.
4.2. Beam-Splitter-Based Approach. A beam splitter is an
optical device that reflects half of the incoming light andtransmits the other half. There are few researches concerningpro-cam systems using a beam splitter. In [7], two cube
beam splitters are used to construct an IR projector and amulti-band camera. However, the beam splitters are used
for internal hardware architecture purposes not calibration
purposes. Fujii et al. [28] briefly described the idea ofscene-independent geometric calibration using a plate beam
splitter attached to an oﬀ-the-shelf projector. The calibration
technique proposed in this section was inspired by this work.
Both their research and ours operate in the visible light
spectrum; however, the camera settings are completely diﬀerent. In their research, a camera uses a normal shutter speed
and works independently to a projector. In our research, as
described in Section 3, the camera is accurately synchronizedwith the DLP projector and its shutter is opened for only
0.55 ms. In this section, we investigate a concrete portable
design and suitability of the beam splitter regarding ournonintrusive projection approach. Many related factors are
introduced and observed. The beam splitter used in ourconfiguration is a TechSpec plate beam splitter 48904-J,
whose dimensions are 75 × 75 mm.
Using a beam splitter to coaxialize the two devices
ensures that any surface visible to a camera can also be
projected upon. The shapes of the surface do not aﬀect
the geometric mapping or cause parallax between the two−→coordinates. This means that geometric mapping ( h ) needs
to be computed only once using (3); recomputation is notnecessary if there is no change in the relative positions ororientations among the projector, camera and beam splitter.
The coaxial concept is illustrated in Figure 7(a) and our beam
splitter configuration is shown in Figure 7(d). The distancefrom the front edge of the wooden base to the projector
lens is 13 cm. In addition to the design proposed in [28],we added a curtain made from a light absorbing black-out
material to achieve the more practical portable design. Thiscurtain not only eliminates reflections of the environment at
the left side of the projector (as shown in Figure 7(c)) but
also prevents the projector’s reflected light from interferingwith the environment (as shown in Figure 7(b)). Note thatthe camera was not in the high shutter speed mode when
capturing Figure 7(c).
As explained in Section 3, the nonintrusive projection
shortens the exposure time of the camera significantly.
Furthermore, the beam splitter setup allows only half ofthe projected light to be transmitted to the surface. As aresult, the amount of light passing through the camera lensin this configuration is quite limited and may result ininaccurate environment analyses. Therefore, we conducted
three experiments using the camera setting explained in
Section 3 and investigated factors related to the practicality of8International Journal of Digital Multimedia Broadcastingabc
VGA splitter

Firewire camera

DLP projector

Curtaind

Plate beam splitter

Figure 7: (a) The concept of pro-cam coaxialization using a beam
splitter. (b) The use of a curtain to block projector’s reflected light.
(c) Captured image without and with the curtain (left and right).
(d) Our beam splitter configuration.

by the camera to about 71.52% of its original intensities
compared with the conventional pro-cam setup. This meansthat amount of light seen by the camera will be quite limitedwith this configuration. Therefore, a projector model whosebrightness is not strong enough might be diﬃcult to be usedin our proposed system.
We investigated the distance in the second experiment.
Because the exposure time of the camera is very short, thefarther away the surface is, the less light the camera willsee. If an environment is insuﬃciently illuminated, it will
be diﬃcult to use any image processing techniques. In thisexperiment, we constantly projected a red image (whose
red intensity value was 255) onto a whiteboard located at
diﬀerent distances. Figure 9 shows the experimental results.
At a distance of 20 cm, the environment was illuminated witha very bright red light. At a distance of 130 cm, the environment was too dark for the camera to see properly. Note thatthe brightness of our DLP projector is 1400 ANSI lumens andthe distance written in this context refers to a distance fromthe surface to the front edge of the wooden base.
Finally, we performed an experiment to measure themaximum distance over which our configuration can perform environment analyses correctly. We projected a pure
red image (with red intensity values equal to 255) at diﬀerent
distances from 20 to 130 cm. The surface was a whiteboard
containing five color magnets inside the projection area. We
applied 2D Gabor filters [29] to each captured image toevaluate the accuracy of the object detection at each distance.
The Gabor filters failed to detect all objects at a distance of
130 cm. Because the projected color used here is the brightest
color possible to be seen by the camera, we conclude that the
maximum distance at which this beam splitter configuration
can be used in the nonintrusive projection mode is 120 cm.

5. Experimental Results and Evaluations
this beam splitter configuration. Only red value is concernedin our experiments because the projector projects the red
light at the selected timeslot.
In the first experiment, we investigated how much thebeam splitter reduces the amount of light seen by the camera.
The three experimental setups are (1) no beam splitter, andcamera directly sees a surface, (2) a beam splitter in frontof the projector lens, but the camera still sees a surface
directly, and (3) our configuration as shown in Figure 7(d).
With a distance of 50 cm from the wooden base to thesurface, we projected uniform red images with intensity
values ranging from 0 to 255 onto a whiteboard, and allowed
the camera to sense these red projections. Figure 8 shows
the red intensities seen by the camera in the three setups.
Comparing the first and second setup, the red intensities
seen by the camera are reduced by 53.78% when the beam
splitter is placed in front of the projector lens. Comparing
the second and third setup, the red intensities are reduced
by 38.39% when the camera sees the surface through thebeam splitter. In total, by comparing the first and third
setup, our configuration reduces the red intensities seenIn this section, we discuss the experiments conducted toevaluate the proposed approaches. All experiments were
performed using a Dell Inspiron 1150 Mobile Intel Pentium 4
laptop with a processor running at 2.80 GHz. The projector’s
focus was adjusted manually in all experiments.
For the nonintrusive projection, we evaluated whether
our sampled colors (from the white set) can illuminate
an environment enough for an environment analysis. At a
specific distance, we projected each color from the white
set onto the whiteboard holding five color magnets, andallowed the camera to sense the whiteboard. After applying
2D Gabor filters to the captured images, the projected colors
causing incorrect detection were counted. Table 1 shows theexperimental results. Most of the misdetection was caused bynoises added in the dark environment. Overall, five magnets
were detected satisfactorily until the distance reached 110 cm.
To determine the accuracy of the pro-cam geometriccalibration on planar surfaces, we measured the geometricerror of the three approaches compared with a ground truth.
Apart from the two proposed approaches (see Section 4),
we added the single calibrated approach for comparison
purposes; this approach lets the camera sense a surface

International Journal of Digital Multimedia Broadcasting

9

#1
#2
#3

255

191

128

64

0

#1

#2

(a)

#3

(b)

Distance fromthe wooden base to the surface (cm)Figure 8: Red intensities seen by the camera in three diﬀerent setups.

130
120
110
100
90
80
70
60
50
40
30
20

16.7787
18.0465
19.7148
21.427124.021226.755930.311336.5682
45.154261.8178

20 cm

130 cm

88.9995
0

20

40
60
80
100
120
Red intensities seen by the camera

145.064
140

160

Figure 9: Red intensities seen by the camera at diﬀerent distances to the surface using the beam splitter setup.

Table 1: Accuracy evaluation of the nonintrusive projection
regarding the distance to the surface.
Distance
(cm)
20
30
40
50
60
70

Number of
misdetected colors
0
0
1
1
4
19

Distance
(cm)
80
90
100
110
120
130

Number of
misdetected colors
19
29
39
48
80
111

Note: There are 115 colors in the white set.

directly and assumes a static geometric mapping between
projector and camera coordinates. The motion sensor oﬄine
calibration was performed using 10 sample images containing 16 calibrated points (N = 10 and n = 16 according
to Section 4.1). In the experiments, camera coordinates
generated by the three approaches were compared withactual camera coordinates. Five experiments were conductedwith diﬀerent orientations of the projector for each approachand each experiment was performed using 16 tested points(the number of tested point written here is not the n value
used in the motion sensor oﬄine calibration). Except for the
beam splitter approach, whose device setup is diﬀerent, theexperiments were conducted simultaneously. Note that theresolution of the camera coordinates was 640 × 480 pixels.
According to the experimental results shown in
Figure 10, the beam-splitter-based approach is the mostaccurate and provides the narrowest range of geometricerrors in both axes. For the other two approaches, error
values derived from the same experiment cluster together.
The clustering locations in the five experiments are similar inboth approaches because they were generated from the same
projector orientations. However, the distribution of error
values in the single calibrated approach is wider than that forthe motion-sensor-based approach. Besides, the accuracy of
the two proposed calibration approaches does not fall overtime. The accuracy of the motion-sensor-based approach
may decrease when the current projector orientation diﬀers
significantly from the reference orientation.
In addition to the beam-splitter-based calibration
approach, we conducted the same experiment with 16 tested
points on five nonplanar surfaces that cannot be calibratedusing the other two calibration approaches. Figure 11 showsthe experimental results and the experimental surfaces
captured by a separate camera. Using the beam splitter setup,geometric errors are small even for these diﬃcult surfaces.
Comparing the motion sensor and the beam splitter
setups, the latter provides more precise geometric calibration
and allows calibration on nonplanar surfaces. This is goodfor a portable system where the geometry of a surface isInternational Journal of Digital Multimedia Broadcasting

Motion sensor based approach/planar surface
20
15
10
5
0
−7 −5 −3 −1
1
3
5
7
−10
−15
−20
ΔX (pixel)
Average: ΔX = −0.61, ΔY = −1.26

ΔY (pixel)

20
15
10
5
0
−7 −5 −3 −1 1
3 5 7
−10
−15
−20
ΔX (pixel)
Average: ΔX = 0.98, ΔY = 4.46

ΔY (pixel)

Single calibrated approach/planar surface(a)ΔY (pixel)

10−7Beam splitter based approach/planar surface
20
15
10
5
0
−5 −3 −1
1
3
5

7

−10
−15
−20
ΔX (pixel)
Average: ΔX = 0.83, ΔY = 0.28

(b)

(c)

ΔY (pixel)

Figure 10: Geometric error (camera coordinates) of the single calibrated approach, the motion-sensor-based approach and the beamsplitter-based approach on planar surfaces.

Beam splitter based approach/nonplanar surface
3
2
1
0
−2
−1 0
2
4
−2
−3
−4
−5
−6
ΔX (pixel)
Average: ΔX = 1.69, ΔY = −1.34

6

(a)

(b)

Figure 11: Geometric error (camera coordinates) of the beam-splitter-based approach on nonplanar surfaces.

not known. However, considering the quality of captured
images, direct capturing can sense the environment more
eﬀectively than indirect capturing. Therefore, by using the
motion sensor setup, image processing can be performed
more eﬃciently.
We measured the geometric error of the perspective
transformation by reprojecting known points onto an
actual surface. Using a planar surface, three experiments
were conducted using the geometric mapping created from4, 9, and 16 correspondences, respectively. During eachexperiment, a chessboard pattern with 25 inner corners
was projected. We located each inner corner in camera
coordinates, applied the perspective transformation to map
those camera coordinates to projector coordinates using(3), and drew the transformed projector coordinates backon the projected images. In this way, we can measure thegeometric error between the actual projector coordinates
and the transformed projector coordinates of each tested
point on the actual surface. Figure 12 shows the experimental
results. While experimenting, we moved neither the surface

nor the devices, and the projection area appearing on thesurface had the dimensions 294 × 222 mm.
Finally, we built a small-scale application using thebeam splitter setup calibrated with 16 correspondences.
The purpose was to demonstrate the entire concept of ourinteractive portable projector. Any objects are detected using
2D Gabor filters; nonintrusive projection is used to draw
text or special eﬀects upon the detected objects or otherareas of the surface. Large-scale programs using the sameprocedure can be built for live broadcasting of virtual studio
productions. Instead of the Gabor detector, human detector,
gesture recognition or motion analysis algorithms might beused to detect an actor and interpret his actions accurately
in real time; robustness of the system regarding moving
targets will depend on the selected detection algorithm. The
rendering technique might be used to virtually paint thestudio, and animation techniques could be utilized to draw
attractive characters interacting with the detected actor.
Figure 13 shows the application in action, including
snapshots of the surface (taken with a separate camera)

International Journal of Digital Multimedia Broadcasting

0
−1

0

1

2

3

−1

ΔY (mm)ΔY (mm)−1

Perspective transformation
error/16 correspondences

Perspective transformation
error/9 correspondences

−2
−3
−4

0
−1

0

1

2

3

−1

ΔY (mm)

Perspective transformation
error/4 correspondences

11

−2
−3

0
−1

0

1

2

3

−2
−3
−4

−4

ΔX (mm)Average: ΔX = 1.58, ΔY = −1.92

ΔX (mm)Average: ΔX = 0.26, ΔY = −0.02

ΔX (mm)Average: ΔX = 0.28, ΔY = 0.3

(a)

(b)

(c)

Figure 12: Geometric error of perspective transformation on an actual surface (world coordinates) using the number of calibrated
correspondences of 4, 9, and 16.

Figure 13: Demonstration of the ipProjector concept. Left images are snapshots of the surface taken by a separated camera. Right images
are images captured by the system camera.

and images captured by the system camera. The numberof detected objects is indicated in red while the otherdetected objects are marked with a “+” sign at the detected
centroid. The “+” signs are drawn in diﬀerent colors but
our separate camera cannot sense them properly. The reader
is encouraged to look closely at the images on the right.
It can be seen that the system camera sensed traces of the
projected contents in some images. This is due to the range ofthe threshold set during the color classification (Section 3.3).
However, these traces are not clear enough to intrude on any
environment analysis. Note that the captured images wereenhanced here for better visualization.

6. Conclusion and Future WorkIn this paper, we investigated hardware setups and software
techniques for creating a real-time interactive projection
application, including live broadcasting from an advanced
virtual studio. The projection device developed in this paper
is self-contained and portable, and can be installed or usedeasily in an existing studio environment. The nonintrusive
projection problem is solved by using an oﬀ-the-shelf DLP
projector together with DLP color wheel analysis, color
space sampling and approximate nearest-neighbor search.

The proposed solution ensures that the environment isalways illuminated, while projected content does not intrude
on any environment analysis. For the real-time geometriccalibration problem, two approaches with diﬀerent setups
were proposed. On a planar surface, the motion-sensorbased approach can update the geometric mapping in realtime. For a more accurate calibration approach that canbe applied to planar and nonplanar surfaces, we proposed
the beam-splitter-based setup. Related factors of this setup
regarding the nonintrusive projection and further image
processing were also investigated.
The device and techniques proposed in this paper will
help the creation of special eﬀects that appear in response toactors or other studio objects in real time. By integrating the
proposed concepts with appropriate target detection algorithms and special eﬀect rendering techniques, an existing
virtual studio setup will then support live broadcasting inan actual studio production. For example, in the weather
forecast virtual studio, positions of the actor and his hands
should be detected so that the weather map can be renderedand projected in response to those detected positions. In
the future, we plan to create a robust interactive projection
application that benefits from the proposed device andtechniques. Furthermore, we are interested in adding human

12detection or hand gesture recognition to the system toincorporate the system with a real human.

References[1] A. Grundhöfer, M. Seeger, F. Häntsch, and O. Bimber,“Dynamic adaptation of projected imperceptible codes,” inProceedings of the 6th IEEE and ACM International Symposiumon Mixed and Augmented Reality (ISMAR ’07), 2007.
[2] D. Cotting and M. Gross, “Interactive environment-aware
display bubbles,” in Proceedings of the Annual ACM Symposium
on User Interface Softaware and Technology (UIST ’06), pp.
245–254, 2006.
[3] X. Cao and R. Balakrishnan, “Interacting with dynamicallydefined information spaces using a handheld projector and apen,” in Proceedings of the Annual ACM Symposium on User
Interface Softaware and Technology (UIST ’06), pp. 225–234,
2006.
[4] X. Cao, C. Forlines, and R. Balakrishnan, “Multi-user interaction using handheld projectors,” in Proceedings of the Annual
ACM Symposium on User Interface Softaware and Technology(UIST ’07), pp. 43–52, 2007.
[5] K. Hosoi, V. N. Dao, and M. Sugimoto, “CoGAME: manipulation by projection,” in Proceedings of the International
Conference on Computer Graphics and Interactive Techniques,
emerging technologies (SIGGRAPH ’07), 2007.
[6] P. Mistry, P. Maes, and L. Chang, “WUW—Wear ur world—a wearable gestural interface,” in Proceedings of the Conferenceon Human Factors in Computing Systems (CHI ’09), pp. 4111–
4116, 2009.
[7] K. Akasaka, R. Sagawa, and Y. Yagi, “A sensor for simultaneously capturing texture and shape by projecting structured
infrared light,” in Proceedings of the 6th International Conference on 3-D Digital Imaging and Modeling (3DIM ’07), pp.
375–381, 2007.
[8] J. C. Lee, S. Hudson, and P. Dietz, “Hybrid infrared andvisible light projection for location tracking,” in Proceedings of
the Annual ACM Symposium on User Interface Softaware andTechnology (UIST ’07), pp. 57–60, 2007.
[9] R. Raskar, G. Welch, M. Cutts, A. Lake, L. Stesin, and H. Fuchs,“The oﬃce of the future: a unified approach to image-basedmodeling and spatially immersive displays,” in Proceedingsof the 25th Annual Conference on Computer Graphics andInteractive Techniques (SIGGRAPH ’98), 1998.
[10] H. Park, M. H. Lee, B. K. Seo, Y. Jin, and J. I. Park, “Content
adaptive embedding of complementary patterns for nonintrusive direct-projected augmented reality,” in Proceedingsof the 12th International Conference on Human-Computer
Interaction (HCI ’07), 2007.
[11] M. Waschbüsch, S. Würmlin, D. Cotting, F. Sadlo, and
M. Gross, “Scalable 3D video of dynamic scenes,” VisualComputer, vol. 21, no. 8–10, pp. 629–638, 2005.
[12] D. Cotting, M. Naef, M. Gross, and H. Fuchs, “Embedding
imperceptible patterns into projected images for simultaneous
acquisition and display,” in Proceedings of the 3rd IEEE andACM International Symposium on Mixed and Augmented
Reality (ISMAR ’04), pp. 100–109, 2004.
[13] S. Borkowski, O. Riﬀ, and J. L. Crowley, “Projecting rectified
images in an augmented environment,” in Proceedings ofthe IEEE International Workshop on Projector-Camera Systems(PROCAMS ’03), 2003.

International Journal of Digital Multimedia Broadcasting[14] M. Fiala, “Automatic projector calibration using selfidentifying patterns,” in Proceedings of the IEEE Computer
Society Conference on Computer Vision and Pattern Recognition(CVPR ’05), 2005.
[15] R. Raskar, J. van Baar, and J. X. Chai, “A low-cost projector
mosaic with fast registration,” in Proceedings of the IEEE Asian
Conference on Computer Vision (ACCV ’02), 2002.
[16] R. Raskar and P. Beardsley, “A self-correcting projector,”in Proceedings of the IEEE Computer Society Conference on
Computer Vision and Pattern Recognition (CVPR ’01), vol. 2,
pp. 504–508, 2001.
[17] J. van Baar, T. Willwacher, S. Rao, and R. Raskar, “Seamless
multi-projector display on curved screens,” in Proceedings ofthe Workshop on Virtual Environments (EGVE ’03), 2003.
[18] R. Raskar, J. van Baar, P. Beardsley, T. Willwacher, S. Rao, andC. Forlines, “iLamps: geometrically aware and self-configuring
projectors,” in Proceedings of the ACM Transactions on Graphics(SIGGRAPH ’03), no. 3, pp. 809–818, July 2003.
[19] W. Sun, X. Yang, S. Xiao, and W. Hu, “Robust checkerboard
recognition for eﬃcient nonplanar geometry registration in
Projector-Camera systems,” in Proceedings of the ACM/IEEE
5th International Workshop on Projector Camera Systems(PROCAMS ’08), 2008.
[20] Y. Ding, J. Xiao, K.-H. Tan, and J. Yu, “Catadioptric projectors,” in Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops(CVPR ’09), pp. 2528–2535, 2009.
[21] A. Kushal, J. van Baar, R. Raskar, and P. Beardsley, “A handheld
projector supported by computer vision,” in Proceedings of the
7th Asian Conference on Computer Vision (ACCV ’06), pp. 183–
192, 2006.
[22] T. Johnson and H. Fuchs, “Real-time projector tracking on
complex geometry using ordinary imagery,” in Proceedingsof the International Workshop on Projector-Camera Systems(PROCAMS ’07), 2007.
[23] S. J. Koppal and S. G. Narasimhan, “Illustrating motion
through DLP photography,” in Proceedings of the IEEE
International Workshop on Projector-Camera Systems (PROCAMS ’03), pp. 9–16, 2009.
[24] S. G. Narasimhan, S. J. Koppal, and S. Yamazaki, “Temporaldithering of illumination for fast active vision,” in Proceedings of the 10th European Conference on Computer Vision(ECCV ’08), pp. 830–844, 2008.
[25] J. S. Beis and D. G. Lowe, “Shape indexing using approximate
nearest-neighbour search in high-dimensional spaces,” in Proceedings of the IEEE Computer Society Conference on Computer
Vision and Pattern Recognition (CVPR ’97), pp. 1000–1006,
1997.
[26] R. Sukthankar, R. G. Stockton, and M. D. Mullin, “Smarter
presentations: exploiting homography in camera-projector
systems,” in Proceedings of the IEEE International Conference
on Computer Vision (ICCV ’01), vol. 1, pp. 247–253, 2001.
[27] V. N. Dao, K. Hosoi, and M. Sugimoto, “A semi-automatic
realtime calibration technique for a handheld projector,” inProceedings of the ACM Symposium on Virtual Reality Software
and Technology (VRST ’07), 2007.
[28] K. Fujii, M. D. Grossberg, and S. K. Nayar, “A projectorcamera system with real-time photometric adaptation for
dynamic environments,” in Proceedings of IEEE Computer
Society Conference on Computer Vision and Pattern Recognition(CVPR ’05), vol. 1, pp. 814–821, 2005.
[29] S. E. Grigorescu, N. Petkov, and P. Kruizinga, “Comparison of
texture features based on Gabor filters,” IEEE Transactions on
Image Processing, vol. 11, no. 10, pp. 1160–1167, 2002.