2013 International Conference on Cyberworlds

Detecting Markers in Blurred and Defocused Images
Masahiro Toyoura
University of Yamanashi
Kofu, Yamanashi, Japan
Email: mtoyoura@yamanashi.ac.jp

Haruhito Aruga
University of Yamanashi
Kofu, Yamanashi, Japan
Email: aruga@yamanashi.ac.jp

Matthew Turk
University of California, Santa Barbara
California, USA
Email: mturk@cs.ucsb.edu

Xiaoyang Mao
University of Yamanashi
Kofu, Yamanashi, Japan
Email: mao@yamanashi.ac.jp

the high-frequency components are attenuated in blurred ordefocused images.
The problem can be solved with the marker having acharacteristic frequency spectrum. In this paper, we describe
the methods for making the new marker and detecting it inblurred and defocused images. We call the marker a monospectrum marker. The key idea is that the mono-spectrum
marker consists of only low-frequency components. The
frequency components pass through the low-pass filteringof blurring and defocusing with a small loss. The markers
can be detected in real time with GPUs by analyzing theirfrequency components. For optimal performance under both
static and dynamic conditions, the mono-spectrum marker
may be combined with current markers. Our approach hasa relatively high computational cost and thus requires GPU
computing. We believe the rapidly growing availability ofGPUs on mobile devices will make this approach to dealing
with blurred and defocused images feasible.

Abstract—Planar markers enable an augmented reality (AR)system to estimate the pose of objects from images containing
them. However, conventional markers are difficult to detectin blurred or defocused images. We propose a new marker
and a new detection and identification method that is designed
to work under such conditions. The problem of conventional
markers is that their patterns consist of high-frequency components such as sharp edges which are attenuated in blurredor defocused images. Our marker consists of a single lowfrequency component. We call it a mono-spectrum marker.
The mono-spectrum marker can be detected in real time witha GPU. In experiments, we confirm that the mono-spectrum
marker can be accurately detected in blurred and defocused
images in real time. Using these markers can increase theperformance and robustness of AR systems and other visionapplications that require detection or tracking of definedmarkers.
Keywords-augmented reality; spectrum analysis; planarmarker;I. I NTRODUCTION

II. R ELATED W ORKS

Estimating the relative position and orientation of a camera from images is a fundamental requirement of augmented
reality (AR). The technique is used for table top interfaces[9], registration for AR surgery systems [4] and many otherapplications, such as estimating robot position and postureor tracking of moving objects. A planar marker is an efficientand effective tool for this estimation in many cases.
However, these applications tend to produce images thatare blurred due to camera motion or defocused due to variations in scene depth with a fixed focal distance, and these
artifacts cause problems with typical marker detection andidentification methods. We propose a new planar marker formobile cameras and moving objects which can be reliably
detected even in blurred or defocused images.
Conventional markers, such as those used in ARToolKit[8] and QR Code, have patterns with high-frequency components such as edges or corners. These attributes make theirmarkers hard to detect in blurred or defocused images, since
978-1-4799-2245-1/13 $31.00 © 2013 IEEE
DOI 10.1109/CW.2013.58

The physical marker of ARToolKit [8] is a popular toolfor calibration, estimating the pose of the camera relative tothe scene. The marker consists of a black bold frame andan internal binary pattern. It is detected in camera images,
and its position and orientation are estimated by locating thefour corners of the black frame. The internal pattern helps
to identify the marker. Other markers, such as the ARTag[5], QR Code, and random dot markers [18] also consist ofbinary edges, corners or dots of black and white.
These conventional markers are difficult to detect andidentify under conditions of image blur and defocusing,which attenuates the high-frequency components from their
sharp edges and corners. Fiala [5] reported how much
Gaussian noise affects marker detection, but did not address
the effect of image blurring and defocusing.
Changes in marker appearance are considered in thenested marker [17] and Bokode tag [11]. The nested marker
has a fractal structure so that it looks the same at long183

cameras cannot be effectively restored. Note that our marker
can be used together with the restoration methods.
Blurring and defocusing parameters are used for displaying virtual objects on the observed images [13]. The parameters are extracted from the appearance changes of the special
pattern marker of ARToolKit under the assumption that theblurring and defocusing are not large. The appearance of thevirtual object is adjusted with the extracted parameters. Our
marker can be used to estimate parameters from blurrier andmore defocused images, making the appearance adjustment
method more usable.

distance as it does at short distance, to provide distanceinvariance. The Bokode tag consists of a light-emitting
component, which is best detected when the camerafs focal
length is set to infinity. These markers do not consider the
possible loss of high-frequency components in images.
Natural feature points like SIFT [10] and SURF [3] alsoenable one to do calibration. The feature points are detected
not in planar markers but in general objects. The systems like
PTAM [9] or DTAM [12] for registration of real environment
and virtual objects are constructed based on the map ofnatural features. The applications are different from those ofplanar markers, since the feature points of the planar markers
are registered in a database in advance in order to estimate
the position and posture of target objects. Like our marker,
SIFT and SURF were designed with frequency changes inmind. They provide frequency-shift-free tracking. However,they do not consider the loss of high-frequency components
altogether, but rather just changes in frequency components
at least when the environmental map is reconstructed. The
Fourier tag [20] and fiducial image [14] use frequency
components for identification, but the components do notcontribute to robust detection or identification in blurred ordefocused images.
We also proposed a new AR marker in [1], but this marker
could not be correctly detected when viewed with a large
distortion caused by perspective projection when the plane
containing the marker is tilted with respect to the camera
plane. The estimated position of a marker in blurred ordefocused images was also less accurate than in images freeof blurring or defocusing; these problems are addressed in
the current paper by introduction of vertical and horizontal
filters and combined use of mono-spectrum marker with the
ARToolKit marker. The details are described in Section III.
Several template matching methods can deal with the blurring and defocusing problem. ESM-Blur [15] creates blurred
templates in advance. Image resampling [6] addresses the
problem by sequentially updating templates. The practical
disadvantage of the template matching method is that only
a few kinds of markers can be used. Conventional markers
for AR support hundreds of markers or more, as does our
mono-spectrum marker.
The failure of marker detection can be mitigated byinterpolating the results of detection in frames [19]. The
method assumes that the detection failure occurs only in a
few frame, but this assumption is not valid when the images
are blurred or defocused.
The recovery of image quality is being studied actively[2], [16]. Blurred or defocused images are degraded images,
and the aim is to restore such images so that they are
free of blurring and defocusing. The degradation is modeledwith a point spread function (PSF). The images are restoredby estimating the PSF and its inverse function. However,restoration works well only when the blurring and defocusing are small. That means images taken with actively moving

III. M ONO - SPECTRUM M ARKER D ESIGN AND
D ETECTION
A. Inhibition of High-Frequency Components by Image
Blurring and Defocusing
Images are blurred when the relative position between
the camera and an observed object changes during exposure.
Blur happens when either the camera or the object is moving.
Although a shorter exposure time can solve the problem insome cases, it tends to produce underlit, and thus noisy,
images.
Figure 1(d) depicts the frequency spectrum of the observed image shown in Figure 1(a). With the camera moving
horizontally, we obtain a blurred image Figure 1(b), and its
spectrum changes as in Figure 1(e). The spectrum loses thehigh-frequency components along the x-axis.

(a) Original image(b) Blurred image(c) Defocused image(d) Spectrum of (a)(e) Spectrum of (b)(f) Spectrum of (c)Figure 1.

Change of frequency spectrum due to blurring and defocusing.

A defocused image is shown in Figure 1(c); its spectrum
changes can be seen in Figure 1(f). The spectrum loses highfrequency components in all directions.
Short exposure time, adequate lighting, stable cameras andobjects, and appropriate focal length would solve the above
problem; however, these measures are impractical in mostdynamic situations, especially in AR applications. Moving
cameras often result in blurred images. Moreover the focal
length is either fixed or adapts to one of many possibly
scene objects. Conventional markers are designed without
consideration of the circumstances where high-frequency

184



components are lost. They are difficult to detect in blurredor defocused images.

V

We present the design of our marker, which is designed to
be invariant to the blurring and defocusing artifacts describedabove. The extraction method covers both detection andidentification of the marker in the observed images.
The mono-spectrum marker consists of the pattern shownin Figure 2(a), a two-dimensional sinusoidal intensity patternwith multiple colors. We designed the marker so that its
brightness changes at a single low frequency. This lowfrequency component is little-affected by blurring and defocusing. Hence, the regions in the observed images corresponding to the markers also have a single low-frequency
component.

marker(b) Combined

Figure 3 shows an overview of marker detection andidentification. We remove the zero frequency components(Step 1) and band-pass filtering of each region (Step 2) in theobserved images. The corner dots are detected by referringto the colors of the regions (Step 3). Valid corner dots are
selected by a segmentation of the original image. The monospectrum markers are then detected and identified (Step 4).
We describe each step in detail below.

(c) (b) in grayscale

Step1. Remove Zero Frequency Component

H

x

V

Let us denote the signal of mono-spectrum markers as
f (x) and the ones of other objects as g(x).
Geometric transform together with a perspective projection changes not only the frequency but also the amplitudeof the signal. The illumination also adds an offset to thesignal. Therefore the signal of the observed marker usually
has a wider range in the frequency domain as shown in theleft column of Figure 3. In the regions corresponding to themono-spectrum marker M and the ones of the other regions
N, the following signals of different features are observed
since the signals are projected on the images.

xx(d) Selected colors
Figure 2.

(1)

C. Extraction of Mono-spectrum Marker

markerColor for corners(1 − α ) cos 2Lπ r + α , r ≤ L20,(otherwise)

where α is the offset of the brightness. By adjusting α , wecan give the center of each dot a brightness which can retain
in both the printed paper and captured image. An appropriate
value for α can be found by checking whether the dots areclearly perceivable in the printed paper, like the one shown in
Figure 2(a). We set α = 0.1 in experiments. Image blurringand defocusing can be assumed to be low-pass filters with acertain cutoff frequency value. The frequency componentsof the center of the regions should be smaller than the
cutoff frequency value. If they are filtered out by blurringor defocusing, the marker detection will fail.

B. Design of the Mono-spectrum Marker(a) Mono-spectrum=(e) Color in HSV color space

Design of the mono-spectrum marker.

The color value of each position of the mono-spectrum
marker is given in hue, saturation, and value (HSV) color
space. The values of H, S and V are determined as follows.
A special H value, yellow in our experiment, is reserved
for the corner dots of the marker. That is, colors excluding
yellow are used for dots that are not corners. The markers
are identified through the alignment of the colors of dots.
Individual colors should be distinguishable even when they
are slightly changed in the images. We empirically used
6 colors for making markers as shown in Figure 2(d).
More number of colors will not be identified in various
illumination conditions.
The value of S is set to the maximum value for eachposition.
The value of V changes with the single low-frequency
component as shown in Figure 2(e). The regions of themarkers are distinguished from other regions by using thecharacteristic frequency spectrum. Let us denote by r the
distance from the center of region, and L the width andheight of a square region. The value of V ∈ [0, 1] is defined
as follows:M:N:f˜(x) + c,g̃(x) + c.

(2)
(3)

f˜ and g̃ means that f and g are transformed with aperspective projection. c is the DC (Direct Current) offset.
In Step 1, we remove the DC components from theoriginal signals. The high pass filter b0 (x) that cuts off only
the DC component is convolved with the observed signals.
M:( f˜(x) + c) ∗ b0 (x) ≈ f˜(x),N:(g̃(x) + c) ∗ b0 (x).

(4)
(5)

Since f (x) does not have a DC component, f˜(x), which
has a transformed single frequency and a little wider range,
should not have it as well. Thus, f˜(x) passes through the185

Mono-spectrum marker
and it spectrum

Horizontal 1D
Band-pass filters

Observed image

Non-DC signal

Marker regions

Marker regions

Other regions

Other regions

Regions passing
through a particularfilter with small loss

Vertical 1D
Band-pass filters
2. Band-pass filtering

1. Remove direct current component

Filtered with H and S
color values

3.3.Corner
Cornerdetection
detection
Divided segments
in binary image

Marker detection in
divided segments

Detect points withlocal maxima

4. Detection and identification

Figure 3.

Extraction of mono-spectrum marker from observed images.

filter b0 (x) in M. On the other hand, g(x) would have a DC
component, and g̃(x) should not pass through the filter b0 (x)in N. The GPU convolves the band-pass filters in the spacedomain in real-time as described in Algorithm 1.

Denoting bvi (i ∈ (1, · · · , nv ) = Nv ) and bhj (i ∈(1, · · · , nh ) = Nh ) as vertical and horizontal band-pass filters,
and s(x) as the observed image; the following region R is
calculated as the candidate region of the mono-spectrum
marker.

Algorithm 1 Remove Zero Frequency Component
1: x = threadid.x, y = threadid.y, sum = 0.0.
2: for (xw, yw) ∈ f ilterWindow0 do
3:
sum = sum + s(x + xw, y + yw) ∗ b0 (xw, yw).
4: end for
5: s0(x, y) = sum.

R p = (x, y) (s ∗ b0 )(x, y) > th p ,
Rv =






 (s ∗ b0 ∗ bvi )(x, y) − (s ∗ b0 )(x, y)  < thr ,(x, y)  ∃i ∈ Nv , (s ∗ b0 )(x, y)
Rh =




 (s ∗ b0 ∗ bh )(x, y) − (s ∗ b0 )(x, y) 



j
(x, y)  ∃ j ∈ Nh ,  < thr ,(s ∗ b0 )(x, y)

Step 2. Band-pass Filtering
In the next step, all small regions are band-pass filtered.
Although f (x) is known in the original marker image,f˜(x) is unknown in the observed image. f˜(x) no longer
appears to be mono-spectrum in the observed image becauseof perspective or the inclination of marker plane. In addition,neither the location nor the range in the frequency domain
is known for the marker in the observed image. To solve
the problem, we use multiple vertical and horizontal bandpass filters that have different pass bands. We still can expectthat most frequency components are located within a certain
range of the transformed single frequency at least along thevertical and horizontal lines. There should be one or morevertical and horizontal filters that pass the signal with a small
loss.

R = R p ∩ R v ∩ Rh ,(6)

where th p is the threshold of absolute brightness and thr
is the threshold of relative remaining power. In experiment,th p was set to 0.05 and thr was set to 0.20. The thresholds
should be appropriately defined depending on the average
brightness of captured images.
The band-pass filters bvi and bhj are designed in thefrequency domain. The shape of each filter is determinedby a particular band to be passed. An inverse Fouriertransformation generates bvi and bhj in the space domain. The
filter passing the signal with period of N1 pixels to N2 pixels
in the space domain is designed such that values are 1 for186marker in Figure 2(a) has 3 × 3 dots. The H values of thedots are put on a lattice that is defined by the four corners.
The H values represent the marker pattern. The marker in
the image is identified by matching it with stored patterns
of markers.
After the identification, the centers of the four corners
are refined by parabola fitting [7]. The brightness peaks ofparabolas define final positions of the corner dots.

1/N2 to 1/N1 in the frequency domain.
The pseudo-code for detecting the candidate regions of
the mono-spectrum markers is shown in Algorithm 2.
Algorithm 2 Detect marker candidate regions
1: candidateRegion = φ .
2: x = threadid.x, y = threadid.y.
3: if s0(x, y) > th p then4:
for i ∈ N do
5:sumv = 0.0.
6:
for xw ∈ f ilterWindowvi do
7:sumv = sumv + s0(x + xw, y) ∗ bvi (xw).
8:
end for9:
if ||sumv − s0(x, y)||/s0(x, y) < thr then10:sumh = 0.0.
11:
for yw ∈ f ilterWindow j do
12:sumh = sumh + s0(x, y + yw) ∗ bhj (yw).
13:
end for14:
if ||sumh − s0(x, y)||/s0(x, y) < thr then15:
candidateRegion = candidateRegion ∪ (x, y).
16:
end if17:
end if18:
end for
19: end if

D. Combined Use with ARToolKit
The position of the marker in images is defined by
the pixels having the local peak of V . Since the local
peaks are detected from one low-frequency component, thedetection is less accurate than the detection from edges orcorners with high-frequency components. For instance, theposition of an ARToolKit marker is defined by the corner
positions. That means ARToolKit’s accuracy is superior to
our mono-spectrum marker when the images are neither
blurred nor defocused. Detecting the peaks from a lowfrequency component tends to be affected by image flicker.
When both marker and camera are stable, ARToolKit gives
better position estimation.
Accordingly, we proceeded to combine our marker with
the ARToolKit marker. An example of such a combination
marker is shown in Figure 2(b). The ARToolKit marker
possesses a black bold frame, and each marker has aunique internal pattern. The marker shown in Figure 2(b)is generated from the one shown in Figure 2(a) with a blackbold frame. Detection and identification of markers are doneusing a binary image. The marker image shown in Figure2(b) is transformed into the grayscale one shown in Figure2(c). When the image is binarized, it becomes an ARToolKit
marker.

Step 3. Corner Detection
The corner dots of the mono-spectrum markers have theselected H values and large S values. The corner dots aredetected by searching for these values in the detected marker
regions in Step 2. The centers of the dots have local maxima.
A window with size corresponding to the filter frequency is
set around each pixel of the marker regions. If a pixel has
the maximum value in its window, the pixel is a candidateof the corner dots.
We avoid false detections of corner dots by first smoothing
the observed images to remove the noise of the observed
images. The set of the candidate corner dots pixels can bealso calculated in real-time with the use of GPU.

Grayscale patch

Confidence value of
corner detection

Binarization

Step 4. Detection and Identification
Valid corner dots are selected from candidates of detected
corner dots in this step. The binarized image of the original
image is divided into segments. The number and positionsof candidates are checked in each segment. If three or fewer
candidates are detected in a segment, these candidates are
removed. The ARToolKit method removes falsely detected
corners in a similar manner. The method can detect multiple
markers in an observed image.
The centers of the four corner dots give a homography
which projects from the marker coordinates to the coordinates of the observed image. The position and posture of themarker are represented with the homography. The pattern ofthe marker is also recognized with the homography. The

Ideal patch

Detected corners

Copy min/max
valuesBinary patch

Figure 4.

Confidence value of corner detection.

The value indicates the amount of blurring or defocusing.
A binarized image is used for corner detection in ARToolKit.
When no blurring and defocusing are observed, the corners
in the original images should be same to the ideal image in
Figure 4. The ideal image is generated by binarised region
painted with minmum and maximum color values in thegrayscale patch of original image. The confidence value is187

Figure 6(a) shows the average error of the extracted four
corners from the blurred images in case that the marker
was extracted using ARToolKit. We assumed that the correct
position of the corners of the marker was the one estimatedwith ARToolKit from the original observed images without
blurring or defocusing. The error was calculated for theestimated position from the blurred or defocused images.
The error is infinity when no marker is detected.
Figure 6(a) shows that the ARToolKit marker and themono-spectrum marker could be detected the all frames.
Since blurring makes it difficult to detect edges perpendicular to the moving direction of the camera, ARToolKit
gave a larger error. The error of the mono-spectrum marker
was not negligible, although it was smaller than that ofARToolKit. The combined use of the mono-spectrum marker
and the ARToolKit marker can accurately extract the marker.
When more blurred as shown in Figure 6(b), the markers
could not be detected in several frames since the lowfrequency components of the mono-spectrum marker wereseverely attenuated with significant blurring. The averageand variance of the error are listed in Figure 7(a). Note thatthe frames of ARToolKit were not used in this case.
Figure 6(c) shows the errors in defocused images. Although the ARToolKit marker could be detected in all
frames, the error was large on average. The mono-spectrum
marker was extracted in all frames, and the error was
smaller than the error of ARToolKit. Image defocusing
makes the ARToolKit marker appear smaller and results
in a larger error. The error of the mono-spectrum marker
is the same level as in the case of image blurring. When
more defocused as shown in Figure 6(d), the ARToolKit
marker was not detected in several frames, since the lowfrequency components were severely attenuated as well as
the case of blurring. The optimal solution is to combine
mono-spectrum markers and ARToolKit markers, as they
have complementary strengths.
Processing times are listed in Figure 7(b). For 640 × 480
pixel images, 15 or more frames were processed per second,so the operation was almost real-time. We used two pairs ofvertical and horizontal band-pass filters for the signal with aperiod of 7 to 15 pixels and the ones with a period of 15 to 31
pixels in the above experiments. More filters gives a wider
depth range of detection, but also uses more processing time.
Consequently, we examined the change in processing timeby varying the number of filters. Since a bandpass filter witha period of 31 to 63 pixels was employed as the third one, the3 × 3 marker can be observed as 189 × 189 pixels. Figure7(b) shows the relationship between the number of pairsof filters and the processing time. For the image resolutionof 640 × 480 pixels, the processing time was 5.3 ms perband-pass filter on average. For 320 × 240 pixel images, theprocessing time was 1.5 ms per band-pass filter on average.
More experimental results are shown in the supplemental
movie. The movie shows the marker detection in various

estimated by summing errors between the grayscale patch
and ideal patch and normalizing the sum.
The threshold of confidence value was set to 0.90 inpractical use. If the value is smaller than the threshold, we
use the estimated position by ARToolKit, and if not, we
use the position of the mono-spectrum marker. Note that
we did not apply the confidence value in the experiment in
the next section, since we aim to show the advantages ofmono-spectrum marker from ARToolKit marker.
IV. E XPERIMENTAL R ESULTS
In order to assess the viability and performance of the
new mono-spectrum marker, we ran two experiments: one
with artificially blurred and defocused images (so we could
control the magnitude of the artifacts), and one with realworld blurred and defocused imagery.
A marker was observed with a USB camera (Qcam S7500
of Logicool Corporation) whose resolution was 640 × 480
pixels and frame rate was 15 fps. We use the auto-whitebalance function of the camera. The marker was detected using a mono-spectrum method as well as ARToolKit methodrunning on a desktop PC (OS: Windows XP, CPU: Intel Core
2 Duo 2.66GHz, GPU: NVidia GeForce GTX295 896MB).
Our program was implemented with CUDA 4.0. Two pairsof vertical and horizontal band-pass filters were used for
detecting the mono-spectrum marker. One pair of the filters
passed a signal with a period of 7 to 15 pixels, and the otherpassed a signal with 15 to 31 pixels. Thus, a 3 × 3 marker
should be observed as 21 to 93 pixels in width and height.
In the first experiment, images s(t) (t = 0, · · · , 510) were
captured with a moving camera. The movement of thecamera was small so that the images would not be blurred.
The focal length of the camera was appropriately set, so theimages would not be defocused.
Next, the images were artificially blurred and defocused in
the following way. Blurred images sB (x,t) were generated bysumming the weighted values in H(−t)G(0, σt2 ) of pixels of
the neighboring frames. H(−t) is the Heaviside function andG(0, σt2 ) is a Gaussian function with variances σt defining
the degree of blurring.
The defocused images sD (x,t) were generated by summing the weighted values in G(0, σx2 ) of pixels of theneighboring pixels with variances σx defining the degree of
defocusing.
The results of marker detection for the blurred images
and the defocused images are shown in Figure 5. The top
images in Figure 5 are the ones of 90th frame in the movie.
Figures 5(b) and 5(c) are examples of blurred images andFigures 5(d) and 5(e) are defocused ones. ARToolKit could
not detect markers in blurred images with σt = 7.0 anddefocused images with σx = 7.0 of Figure 5. ARToolKit
markers tend to be detected on farther positions in defocused
images.

188

illumination scenes and various scales. The images arecaptured by a moving camera. The markers could be robustly
detected even in blurred or defocused images.
We also confirmed the validity of combined use with
ARToolKit in the movie. We captured real blurred anddefocused images and evaluated the detection rate as an ARToolKit marker and mono-spectrum marker. In the blurred
images, ARToolKit marker was missed in 22 images out of
500, and mono-spectrum marker was correctly detected in all
images. In the defocused images, both of ARToolKit marker
and mono-spectrum marker were detected in all images,
although the position of the ARToolKit marker was farther
than the real position.

[6] E. Ito, T. Okatani, and K. Deguchi. Robust planar target
tracking and pose estimation from a single concavity. In
International Symposium on Mixed and Augmented Reality(ISMAR), 2011.
[7] K. Kanatani and Y. Sugaya. Performance evaluation of iterative geometric fitting algorithms. Computational Statistics
and Data Analysis, 52:1208–1222, 2007. Parabola Fitting.
[8] H. Kato and M. Billinghurst. Marker tracking and hmd
calibration for a video-based augmented reality conferencing
system. In International Workshop on Augmented Reality,
pages 85–94, 1999.
[9] G. Klein and D. Murray. Parallel tracking and mapping forsmall ar workspaces. In International Symposium on Mixed
and Augmented Reality (ISMAR), 2007.

V. C ONCLUSIONS[10] D. G. Lowe.
Distinctive image features from scaleinvariant keypoints. International Journal of Computer Vision, 60(2):91–110, 2004.

We proposed a mono-spectrum marker that can be accurately extracted even from blurred or defocused images.
The marker has only a low-frequency component. Band-passfilters that pass different frequency components are used forextracting the mono-spectrum marker. In an experiment, themarker was accurately extracted from blurred or defocused
images in real-time.
Processing without GPUs is also a future topic. GPUs
were used for band-pass filtering. They can filter in parallel
all pixels. ARToolKit has an advantage in processing timeand required PC specs. The need to use GPUs would restrict
the applications of the mono-spectrum marker. The disadvantage could be overcome with digital signal processors(DSPs). DSPs are used in mobile devices for processing
voice signals, and filtering voice signals is a major function.
Such a function could be used for extracting the monospectrum marker. On the other hand, GPUs are also mounted
on into recent mobile devices. They would be useful for
extracting mono-spectrum markers.

[11] A. Mohan, G. Woo, S. Hiura, Q. Smithwick, and R. Raskar.
Bokode: Imperceptible visual tags for camera based interaction from a distance. ACM Transactions on Graphics, 28(3),2009. Article 98.
[12] R. A. Newcombe, S. Lovegrove, and A. J. Davison. Dtam:
Dense tracking and mapping in real-time. In International
Conference on Computer Vision (ICCV), pages 2320–2327,
2011.
[13] B. Okumura, M. Kanbara, and N. Yokoya. Augmented reality
based on estimation of defocusing and motion blurring from
captured images. In International Symposium on Mixed andAugmented Reality (ISMAR), pages 219–225, 2006.
[14] C. Owen, F. Xiao, and P. Middlin. What is the best fiducial? In
IEEE International Workshop on Augmented Reality Toolkit,
2002.
[15] Y. Park, V. Lepetit, and W. Woo. Esm-blur: Handlingand rendering blur in 3d tracking and augmentation. In
International Symposium on Mixed and Augmented Reality(ISMAR), pages 163–166, 2009.

R EFERENCES[1] H. Aruga, M. Toyoura, and X. Mao. A trackable ar marker
in blurred or defocused images. In Meeting on Image Recognition and Understanding (MIRU), pages 496–503, 2011. (InJapanese)[16] Q. Shan, J. Jia, and A. Agarwala. High-quality motion
deblurring from a single image. ACM Transactions on
Graphics, 27(3), 2008.

[2] H. Asai, Y. Oyamada, J. Pilet, and H. Saito. Cepstral
analysis based blind deconvolution for motion blur. In IEEE
International Conference on Image Processing (ICIP), 2010.

[17] K. Tateno, I. Kitahara, and Y. Ohta. A nested marker foraugmented reality. In IEEE Virtual Reality, pages 259–262,
2007.

[3] H. Bay, A. Ess, T. Tuytelaars, and L. V. Gool. Surf:
Speeded up robust features. Computer Vision and Image
Understanding, 110(3):346–359, 2008.

[18] H. Uchiyama and H. Saito. Random dot markers. In IEEE
Virtual Reality (IEEE VR), 2011.
[19] Y. Uematsu and H. Saito. Improvement of accuracy for 2dmarker-based tracking using particle filter. In International
Conference on Artificial Reality and Telexistence (ICAT),
pages 183–189, 2007.

[4] C. Bichlmeier, F. Wimmer, S. Heining, and N. Navab. Contextual anatomic mimesis: Hybrid in-situ visualization method
for improving multi-sensory depth perception in medical
augmented reality. In International Symposium on Mixed andAugmented Reality (ISMAR), pages 129–138, 2007.

[20] A. Xu and G. Dudek. Fourier tag: A smoothly degradable
fiducial marker system with configurable payload capacity. In
Canadian Conference on Computer and Robot Vision (CRV),
pages 40–47, 2011.

[5] M. Fiala. Artag, a fiducial marker system using digitaltechniques. In IEEE Conference on Computer Vision and
Pattern Recognition (CVPR), pages 590–596, 2005.

189

(b) σt = 4.0(a) Original image(c) σt = 7.0(d) σx = 4.0(e) σx = 7.0

Results of marker detection on blurred and defocused images. (Top-left) Original image of 90th frame. (Top) Blurred ((b) and (c)) or defocused

140

120

120

100

100

100

80
60

60

80
60

40

40

40

20

20

20

0

0

100

200
300
Frame

400

0

500

0

100

200
300
Frame

400

0

500

80
60
40
20

0

100

200
300
Frame

400

0

500

140

140

140

120

120

120

120

100

100

100

100

80
60

80
60

80
60

40

40

40

20

20

20

0

0

0

0

100

200
300
Frame

400

500

0

(a) σt = 4.0

100

200
300
Frame

400

500

(b) σt = 7.0

Error (mm)140

Error (mm)Error (mm)80

Error (mm)

140

120

100

Error (mm)

140

120
Error (mm)140

Error (mm)Error (mm)Figure 5.
((d) and (e)) images. (Middle) Teapots placed on the marker positions detected as an ARToolKit marker. The marker was not detected when images were
blurred and defocused, and detected on farther positions when images were defocused. (Bottom) Teapots placed on the marker positions detected as a
mono-spectrum marker. The marker positions were correctly estimated.

0

100

200
300
Frame

400

500

0

100

200
300
Frame

400

500

80
60
40
20

0

100

200
300
Frame

400

500

0

(c) σx = 4.0(d) σx = 7.0

Figure 6. Results of marker detection. (Top) Average errors of four corner positions of detected markers as ARToolKit markers. (Bottom) Errors as
mono-spectrum marker. The x-axis is the number of frames, and the y-axis is the error.
Resolution of images
640 x 480
320 x 240
Ave.
24.3

Var.
356.2

Time
9.2

Ave.
25.6

Var.
364.7

Time
5.0

28.6

10.4

7.9

30.6

113.7

3.8

5.3

19.8

58.3

7.1

76.8

17.1

3.4

2.7

56.8

3.9

3.9

16.2

Processing time
[ms/frame]

ARToolKit marker(Blur, σt =4.0)
ARToolKit marker(Defocus, σt =7.0)
Mono-spectrum marker(Blur, σt =4.0)
Mono-spectrum marker(Defocus, σt =7.0)

Resolution: 640x480
Resolution: 320x240
100
57.2

61.0

67.8

16.1

16.5

19.0

50

0

(a) Errors and processing times (ms/frame) of marker detection.
(*: Not including frames in which markers could not be detected.)
Figure 7.

123
Number of band-pass filters(b) Processing time versus number of band-pass filters.

Errors and processing time.

190