Noname manuscript No.
(will be inserted by the editor)

Mono-spectrum Marker: An AR Marker Robust to Image Blurand Defocus
Masahiro Toyoura · Haruhito Aruga · Matthew Turk · Xiaoyang Mao

Published online: 20 December 2013

Abstract Planar markers enable an augmented reality(AR) system to estimate the pose of objects from images containing them. However, conventional markers
are diﬃcult to detect in blurred or defocused images.
We propose a new marker and a new detection andidentiﬁcation method that is designed to work undersuch conditions. The problem of conventional markers
is that their patterns consist of high-frequency components such as sharp edges which are attenuated inblurred or defocused images. Our marker consists ofa single low-frequency component. We call it a monospectrum marker. The mono-spectrum marker can bedetected in real time with a GPU. In experiments, we
conﬁrm that the mono-spectrum marker can be accurately detected in blurred and defocused images in realtime. Using these markers can increase the performanceand robustness of AR systems and other vision applications that require detection or tracking of deﬁned markers.
Keywords Augmented reality · Spectrum analysis ·Planar marker

1 Introduction
Estimating the relative position and orientation of acamera from images is a fundamental requirement ofaugmented reality (AR). The technique is used for table top interfaces [10], registration for AR surgery sysM. Toyoura · H. Aruga · X. Mao
University of Yamanashi,
4-3-11, Kofu, Yamanashi, 400-8511, Japan
E-mail: mtoyoura@yamanashi.ac.jp
M. Turk
University of California,
Santa Barbara, CA 93106-5110, USA

tems [4] and many other applications, such as estimating robot position and posture or tracking of moving
objects. A planar marker is an eﬃcient and eﬀective
tool for this estimation in many cases.
However, these applications tend to produce images
that are blurred due to camera motion or defocused due
to variations in scene depth with a ﬁxed focal distance,
and these artifacts cause problems with typical marker
detection and identiﬁcation methods. We propose a newplanar marker for mobile cameras and moving objects
which can be reliably detected even in blurred or defocused images.
Conventional markers, such as those used in ARToolKit [9] and QR Code, have patterns with highfrequency components such as edges or corners. These
attributes make their markers hard to detect in blurredor defocused images, which have attenuated high-frequency
components.
The problem can be solved with the marker having a characteristic frequency spectrum. In this paper,
we describe the methods for making the new marker
and detecting it in blurred and defocused images. We
call the marker a mono-spectrum marker. The key idea
is that the mono-spectrum marker consists of only lowfrequency components. The frequency components passthrough the low-pass ﬁltering of blurring and defocusing with a small loss. The markers can be detected inreal time with GPUs by analyzing their frequency components. Our approach has a relatively high computational cost and thus requires GPU computing. We believe the rapidly growing availability of GPUs on mobile
devices will make this approach to dealing with blurredand defocused images feasible.

2

2 Related Works
The physical marker of ARToolKit [9] is a popular toolfor calibration, estimating the pose of the camera relative to the scene. The marker consists of a black bold
frame and an internal binary pattern. It is detected incamera images, and its position and orientation are estimated by locating the four corners of the black frame.
The internal pattern helps to identify the marker. Other
markers, such as the ARTag [6], QR Code, and random
dot markers [19] also consist of binary edges, corners or
dots of black and white.
These conventional markers are diﬃcult to detectand identify under conditions of image blur and defocusing, which attenuates the high-frequency components from their sharp edges and corners. Fiala [6] reported how much Gaussian noise aﬀects marker detection, but did not address the eﬀect of image blurringand defocusing.
Changes in marker appearance are considered in thenested marker [18] and Bokode tag [12]. The nested
marker has a fractal structure so that it looks the sameat long distance as it does at short distance, to provide distance invariance. The Bokode tag consists of alight-emitting component, which is best detected when
the camera’s focal length is set to inﬁnity. These markers do not consider the possible loss of high-frequency
components in images.
Natural feature points like SIFT [11] and SURF [3]also enable one to do calibration. The feature points aredetected not in planar markers but in general objects.
Systems like PTAM [10] or DTAM [13] for registrationof real environment and virtual objects are constructedbased on the map of natural features. The applications
are diﬀerent from those of planar markers, since thefeature points of the planar markers are registered in a
database in advance in order to estimate the positionand posture of target objects. Like our marker, SIFT
and SURF were designed with frequency changes inmind. They provide frequency-shift-free tracking. However, they do not consider the loss of high-frequency
components altogether, but rather just changes in frequency components at least when the environmental
map is reconstructed. The Fourier tag [21] and ﬁducial
image [15] use frequency components for identiﬁcation,
but the components do not contribute to robust detection or identiﬁcation in blurred or defocused images.
We also proposed a new AR marker in [1], but thismarker could not be correctly detected when viewed
with a large distortion caused by perspective projection
when the plane containing the marker is tilted withrespect to the camera plane. The problem is addressedin the current paper by the introduction of vertical and

Masahiro Toyoura et al.

horizontal ﬁlters. The details are described in Section
3.
Several template matching methods can deal withthe blurring and defocusing problem. ESM-Blur [16]creates blurred templates in advance. Image resampling[7] addresses the problem by sequentially updating templates. The practical disadvantage of the template matching method is that only a few kinds of markers can beused. Conventional markers for AR support hundredsof markers or more, as does our mono-spectrum marker.
The failure of marker detection can be mitigated byinterpolating the results of detection in frames [20]. The
method assumes that the detection failure occurs only
in a few frames, but this assumption is not valid when
the images are blurred or defocused.
The recovery of image quality is being studied actively [2, 17]. Blurred or defocused images are degradedimages, and the aim is to restore such images so thatthey are free of blurring and defocusing. The degradation is modeled with a point spread function (PSF).
The images are restored by estimating the PSF and
its inverse function. However, restoration works wellonly when the blurring and defocusing are small. Thatmeans images taken with actively moving cameras cannot be eﬀectively restored. Note that our marker canbe used together with the restoration methods.
Blurring and defocusing parameters are used for displaying virtual objects on the observed images [14]. The
parameters are extracted from the appearance changesof the special pattern marker of ARToolKit under theassumption that the blurring and defocusing are notlarge. The appearance of the virtual object is adjustedwith the extracted parameters. Our marker can be used
to estimate parameters from blurrier and more defocused images, making the appearance adjustment method
more usable.
3 Mono-spectrum Marker Design and
Detection
3.1 Inhibition of High-Frequency Components by
Image Blurring and Defocusing
Images are blurred when the relative position between
the camera and an observed object changes during exposure. Blur happens when either the camera or the object is moving. Although a shorter exposure time cansolve the problem in some cases, it tends to produce
underlit, and thus noisy, images.
Figure 1(b) depicts the frequency spectrum of theobserved image shown in Figure 1(a). With the camera
moving horizontally, we obtain a blurred image Figure1(c), and its spectrum changes as in Figure 1(d). The

Mono-spectrum Marker: An AR Marker Robust to Image Blur and Defocus(a) Original image(b) Spectrum of (a)(c) Blurred image(d) Spectrum of (c)3

The mono-spectrum marker consists of the patternshown in Figure 2, a two-dimensional sinusoidal intensity pattern with multiple colors that looks like a pattern of blurry dots to the observer. We designed themarker so that its brightness changes at a single low frequency. This low-frequency component is little-aﬀectedby blurring and defocusing. Hence, the regions in theobserved images corresponding to the markers also havea single low-frequency component.

H

S

V

Fig. 2 Design of the mono-spectrum marker.
(e) Defocused image(f) Spectrum of (e)Fig. 1 Change of frequency spectrum due to blurring anddefocusing.

spectrum loses the high-frequency components along
the x-axis.
Images are defocused when the focal length of thecamera is incorrectly set for the current distance between the camera and the object. The degree of defocusing depends on the diﬀerence between the focal distanceand the distance between the camera and the object.
Short exposure time, adequate lighting, stable cameras
and objects, and appropriate focal length would solve
the above problem; however, these measures are impractical in most dynamic situations, especially in AR
applications. Moving cameras often result in blurred
images. Moreover, the focal length is either ﬁxed or
adapts to one of many possibly scene objects.
A defocused image is shown in Figure 1(e); its spectrum changes can be seen in Figure 1(f). The spectrumof a defocused image loses high-frequency components
in all directions.
Conventional markers are designed without consideration of the circumstances where high-frequency components are lost. They are diﬃcult to detect in blurredor defocused images.

3.2 Design of the Mono-spectrum Marker
We present the design of our marker, which is designed
to be invariant to the blurring and defocusing artifacts
described above. The extraction method covers both detection and identification of the marker in the observed
images.

The color value of each position of the mono-spectrum
marker is given in hue, saturation, and value (HSV)color space. The values of H, S and V are determined
as follows.
A special H value, yellow in our experiment, is reserved for the corner dots of the marker. That is, colors
excluding yellow are used for dots that are not corners.
The markers are identiﬁed through the alignment of thecolors of dots. Individual colors should be distinguishable even when they are slightly changed in the images. We empirically used 6 colors for making markers.
A larger number of colors will not be distinguishable invarious illumination conditions.
The value of S is set to the maximum value for eachposition.
The value of V changes with the single low-frequency
component. The regions of the markers are distinguishedfrom other regions by using the characteristic frequency
spectrum. Let us denote by r the distance from the center of region, and L the width and height of a squareregion. The value of V ∈ [0, 1] is deﬁned as follows:
{
V =

(
)
(
)
(1 − α) cos 2πr+ α, r ≤ L2
L
0,(otherwise)(1)

where α is the oﬀset of the brightness. By adjusting α,
we can give the center of each dot a brightness which
can retain in both the printed paper and captured image. An appropriate value for α can be found by checking whether the dots are clearly perceivable in the printed
paper, like the one shown in Figure 2. We set α = 0.1
in our experiments. Image blurring and defocusing canbe assumed to be low-pass ﬁlters with a certain cutoﬀ frequency value. The frequency components of the4Masahiro Toyoura et al.

center of the regions should be smaller than the cutoﬀ
frequency value. If they are ﬁltered out by blurring or
defocusing, the marker detection will fail.

3.3 Number of Possible Marker Patterns
Corner dots have a reserved color. Figure 3 shows rotationally identical dots. The dots indicated with thesame shape are rotationally identical for the rotation
angles of 0, 90, 180, and 270 degrees. A group of samecolor dots matches another group of dots when themarker is rotated. A 4 × 4 marker has 4 groups of 3
dots. We denote by d the number of dots on an edge. Incase of the 4×4 marker, d is 4. According to the CauchyFroebenius theorem [5], when d is odd, a marker has 4groups of u1 = (d − 1)(d + 1)/4 − 1 dots together withan independent dot at the center of marker. When d iseven, a marker has 4 groups of u2 = (d/2)2 − 1 dots.
The number of patterns that m colors are assigned to
the groups of dots is calculated as the following.
 ( 4u) m m 1 +m2u1 +2mu1 , (If d is odd)4()
 m4u2 +m2u2 +2mu2 . (If d is even)(2)4(a) 4x4 dot marker(b) 5x5 dot marker

Fig. 3 Groups of dots that are rotationally identical.

The number of patterns is 825 when m = 5 andd = 3, and 544, 207, 356 when m = 6 and d = 4. m andd should be set in consideration of the size of markers,
robustness of marker extraction, and required numberof patterns.

3.4 Extraction of Mono-spectrum Marker
Figure 4 shows an overview of marker detection andidentiﬁcation. We remove the zero frequency components (Step 1) and band-pass ﬁltering of each region(Step 2) in the observed images. The corner dots aredetected by referring to the colors of the regions (Step3). Valid corner dots are selected by a segmentationof the original image. The mono-spectrum markers arethen detected and identiﬁed (Step 4). We describe eachstep in detail below.

Step1. Remove Zero Frequency Component
Let us denote the signal of mono-spectrum markers as
f (x) and that of other objects as g(x).
Geometric transform together with a perspective
projection changes not only the frequency but also theamplitude of the signal. The illumination also adds an
oﬀset to the signal. Therefore the signal of the observed
marker usually has a wider range in the frequency domain as shown in the left column of Figure 4. In theregions corresponding to the mono-spectrum marker M
and the ones of the other regions N , the following signals of diﬀerent features are observed since the signals
are projected on the images:
M : f˜(x) + c,

(3)

N : g̃(x) + c.

(4)f˜ and g̃ means that f and g are transformed with aperspective projection. c is the DC (Direct Current)
oﬀset.
In Step 1, we remove the DC components from theoriginal signals. The high pass ﬁlter b0 (x) that cuts oﬀ
only the DC component is convolved with the observed
signals.
M : (f˜(x) + c) ∗ b0 (x) ≈ f˜(x),(5)

N : (g̃(x) + c) ∗ b0 (x).

(6)

Since f (x) originally consists of a single frequency,f˜(x) should have a transformed single frequency and alittle wider range. Thus, f˜(x) passes through the ﬁlter
b0 (x) in M . On the other hand, g(x) would consist ofvarious frequency components, and g̃(x) should not passthrough the ﬁlter b0 (x) in N . The GPU convolves the
band-pass ﬁlters in the space domain in real-time as
described in Algorithm 1.
Algorithm 1 Remove Zero Frequency Component
1: x = threadid.x, y = threadid.y, sum = 0.0.
2: for (xw, yw) ∈ f ilterW indow0 do
3:
sum = sum + s(x + xw, y + yw) ∗ b0 (xw, yw).
4: end for
5: s0(x, y) = sum.

Step 2. Band-pass Filtering
In the next step, all small regions are band-pass ﬁltered.
Although f (x) is known in the original marker image, f˜(x) is unknown in the observed image. f˜(x) nolonger appears to be mono-spectrum in the observed

Mono-spectrum Marker: An AR Marker Robust to Image Blur and Defocus

5

Mono-spectrum marker
and it spectrum

Horizontal 1D
Band-pass filters

Observed image

Non-DC signal

Marker regions

Marker regions

Other regions

Other regions

Regions passing
through a particularfilter with small loss

Vertical 1D
Band-pass filters
2. Band-pass filtering

1. Remove direct current component

Filtered with H and S
color values

3. Corner detection
Divided segments
in binary image

Marker detection in
divided segments

Detect points withlocal maxima

4. Detection and identification

Fig. 4 Extraction of mono-spectrum marker from observed images.

image because of perspective or the inclination of marker
plane. In addition, neither the location nor the range
in the frequency domain is known for the marker in theobserved image. To solve the problem, we use multiple vertical and horizontal band-pass ﬁlters that havediﬀerent pass bands. We still can expect that most frequency components are located within a certain rangeof the transformed single frequency at least along thevertical and horizontal lines. There should be one ormore vertical and horizontal ﬁlters that pass the signal
with a small loss.
Denoting bvi (i ∈ (1, · · · , nv ) = Nv ) and bhj (i ∈(1, · · · , nh ) = Nh ) as vertical and horizontal band-passﬁlters, and s(x) as the observed image; the following
region R is calculated as the candidate region of themono-spectrum marker.
dif fiv (x, y) =(s ∗ b0 ∗ bvi )(x, y) − (s ∗ b0 )(x, y),(s ∗ b0 )(x, y)

dif fih (x, y) =(s ∗ b0 ∗ bhj )(x, y) − (s ∗ b0 )(x, y),(s ∗ b0 )(x, y)

Rp = {(x, y) |(s ∗ b0 )(x, y) > thp } ,Rv = {(x, y) | ∃i ∈ Nv , dif fiv (x, y) < thr } ,{}Rh = (x, y) ∃j ∈ Nh , dif fih (x, y) < thr ,R = Rp ∩ Rv ∩ Rh ,(7)

where thp is the threshold of absolute brightness andthr is the threshold of relative remaining power. The

thresholds should be appropriately deﬁned dependingon the average brightness of captured images. Autowhite-balance function helps to deﬁne the appropriate
values of thp and thr , and the function is generally implemented in recent consumer cameras. All the experimental results in Section 4 are given with the same
parameters, which thp was set to 0.05 and thr was set
to 0.20.
The band-pass ﬁlters bvi and bhj are designed in thefrequency domain. The shape of each ﬁlter is determined by a particular band to be passed. An inverse
Fourier transformation generates bvi and bhj in the spacedomain. The ﬁlter passing the signal with period of N1
pixels to N2 pixels in the space domain is designed suchthat values are 1 for 1/N2 to 1/N1 in the frequency domain.
The pseudo-code for detecting the candidate regionsof the mono-spectrum markers is shown in Algorithm
2.
Step 3. Corner Detection
The corner dots of the mono-spectrum markers havethe selected H values and large S values. The cornerdots are detected by searching for these values in the detected marker regions in Step 2. The centers of the dots
have local maxima. A window with size corresponding
to the ﬁlter frequency is set around each pixel of themarker regions. If a pixel has the maximum value in its
window, the pixel is a candidate of the corner dots.

6

Algorithm 2 Detect marker candidate regions
1: candidateRegion = ϕ.
2: x = threadid.x, y = threadid.y.
3: if s0(x, y) > thp then4:
for i ∈ N do
5:sumv = 0.0.
6:
for xw ∈ f ilterW indowiv do
7:sumv = sumv + s0(x + xw, y) ∗ bv
i (xw).
8:end for9:
if ||sumv − s0(x, y)||/s0(x, y) < thr then10:sumh = 0.0.
11:
for yw ∈ f ilterW indowj do
12:sumh = sumh + s0(x, y + yw) ∗ bh
j (yw).
13:
end for14:
if ||sumh − s0(x, y)||/s0(x, y) < thr then15:
candidateRegion = candidateRegion ∪ (x, y).
16:
end if17:
end if18:
end for
19: end if

We avoid false detections of corner dots by ﬁrst
smoothing the observed images to remove the noise of
the observed images. The set of the candidate cornerdots pixels can be also calculated in real-time with theuse of the GPU.
Step 4. Detection and Identiﬁcation
Valid corner dots are selected from candidates of detected corner dots in this step. The binarized image ofthe original image is divided into segments. The number and positions of candidates are checked in each segment. If three or fewer candidates are detected in asegment, these candidates are removed. False positive
candidate corners may be extracted especially from acluttered background, counting four corners in a binarized segment guarantees that only the true corners ofmono-spectrum markers are detected. The ARToolKit
method removes falsely detected corners in a similarmanner. The method can detect multiple markers inan observed image.
The centers of the four corner dots give a homography which projects from the marker coordinates to thecoordinates of the observed image. The position andposture of the marker are represented with the homography. The pattern of the marker is also recognized withthe homography. The marker in Figure 2 has 3×3 dots.
The H values of the dots are put on a lattice that isdeﬁned by the four corners. The H values represent themarker pattern. The marker in the image is identiﬁedby matching it with stored patterns of markers.
After the identiﬁcation, the centers of the four corners are reﬁned by parabola ﬁtting [8]. The brightness
peaks of parabolas deﬁne ﬁnal positions of the cornerdots.

Masahiro Toyoura et al.

4 Experimental Results
In order to assess the viability and performance of the
new mono-spectrum marker, we ran two experiments:
one with artiﬁcially blurred and defocused images (sowe could control the magnitude of the artifacts), andone with real-world blurred and defocused imagery.
A marker was observed with a USB camera (Qcam
S7500 of Logicool Corporation) whose resolution was
640 × 480 pixels and frame rate was 15 fps. We used theauto-white-balance function of the camera. The marker
was detected using our mono-spectrum method as wellas ARToolKit method running on a desktop PC (OS:Windows XP, CPU: Intel Core 2 Duo 2.66GHz, GPU:
NVidia GeForce GTX295 896MB). Our program was
implemented with CUDA 4.0. Two pairs of vertical andhorizontal band-pass ﬁlters were used for detecting themono-spectrum marker. One pair of the ﬁlters passed asignal with a period of 7 to 15 pixels, and the otherpassed a signal with 15 to 31 pixels. Thus, a 3 × 3
marker should be observed as 21 to 93 pixels in widthand height.
In the ﬁrst experiment, images s(t) (t = 0, · · · , 510)
were captured with a moving camera. The movementof the camera was small so that the images would not
be blurred. The focal length of the camera was appropriately set, so the images would not be defocused.
Next, the images were artiﬁcially blurred and defocused in the following way. Blurred images sB (x, t) weregenerated by summing the weighted values in H(−t)G(0,σt2 ) of pixels of the neighboring frames. H(−t) is the
Heaviside function and G(0, σt2 ) is a Gaussian function
with variances σt deﬁning the degree of blurring.
The defocused images sD (x, t) were generated bysumming the weighted values in G(0, σx2 ) of pixels of
the neighboring pixels with variances σx deﬁning thedegree of defocusing.
The results of marker detection for the blurred images and the defocused images are shown in Figure
5. The top images in Figure 5 are the ones of 67th
frame in the movie. Figures 5(b) and 5(c) are examples of blurred images and Figures 5(d) and 5(e) aredefocused ones. ARToolKit could not detect markers
in blurred images with σt = 7.0 and defocused images
with σx = 7.0 of Figure 5. ARToolKit markers tend to
be detected on farther positions in defocused images.
The position of mono-spectrum markers were estimatedmore correctly than those of ARToolKit markers.
Figure 6(a) shows the average error of the extracted
four corners from the blurred images when the marker
was extracted using ARToolKit. We assumed that thecorrect position of the corners of the marker was theone estimated with ARToolKit from the original ob-Mono-spectrum Marker: An AR Marker Robust to Image Blur and Defocus

served images without blurring or defocusing. The error was calculated for the estimated position from theblurred or defocused images. The error is inﬁnity when
no marker is detected.
Figure 6(a) shows that the ARToolKit marker andthe mono-spectrum marker could be detected the allframes. Since blurring makes it diﬃcult to detect edges
perpendicular to the moving direction of the camera,
mono-spectrum marker gave a smaller error, although
it was not negligible. When more blurred as shown in
Figure 6(b), the markers could not be detected in several frames since the low-frequency components of themono-spectrum marker were severely attenuated withsigniﬁcant blurring. The average and variance of theerror are listed in Figure 7(a).
Figure 6(c) shows the errors in defocused images.
Although the ARToolKit marker could be detected in
all frames, the error was large on average. The monospectrum marker was extracted in all frames, and theerror was smaller than the error of ARToolKit. Image defocusing makes the ARToolKit marker appear
smaller and results in a larger error. The error of themono-spectrum marker is the same level as in the caseof image blurring. When more defocused as shown in
Figure 6(d), both of markers were not detected in several frames, since the low-frequency components wereseverely attenuated as well as the case of blurring.
Processing times for our mono-spectrum marker approach are listed in Figure 7(b). For 640 × 480 pixel images, 15 or more frames were processed per second, so
the operation was almost real-time. We used two pairsof vertical and horizontal band-pass ﬁlters for the signal
with a period of 7 to 15 pixels and the ones with a period of 15 to 31 pixels in the above experiments. Moreﬁlters gives a wider depth range of detection, but alsouses more processing time. Consequently, we examinedthe change in processing time by varying the number ofﬁlters. Since a bandpass ﬁlter with a period of 31 to 63
pixels was employed as the third one, the 3 × 3 marker
can be observed as 189 × 189 pixels. Figure 7(b) shows
the relationship between the number of pairs of ﬁlters
and the processing time. For the image resolution of
640 × 480 pixels, the processing time was 5.1 ms perband-pass ﬁlter on average. For 320 × 240 pixel images,
the processing time was 1.2 ms per band-pass ﬁlter on
average.
More experimental results are shown in Figure 8.
Figure 8 shows the marker detection in various illumination scenes including in indoor and outdoor, andvarious scales. The images are captured by a moving
camera. The markers could be robustly detected evenin blurred or defocused images. The results shown in
Figure 8 were generated with the parameter setting.

7

The number and supporting range of the ﬁlters do not
require further tuning in the cases. The additional ﬁlter supporting a period of 31 to 63 pixels enables us
to detect a 3 × 3 marker observed as 189 × 189 pixels, although more number of ﬁlters takes more time as
mentioned above.
5 Conclusions
We proposed a mono-spectrum marker that can be accurately extracted even from blurred or defocused images. The marker has only a low-frequency component.
Band-pass ﬁlters that pass diﬀerent frequency components are used for extracting the mono-spectrum marker.
In an experiment, the marker was accurately extracted
from blurred or defocused images in real-time.
Processing without GPUs is also a future topic. GPUs
were used for band-pass ﬁltering. They can ﬁlter in parallel all pixels. ARToolKit has an advantage in processing time and required PC specs. The need to use GPUs
would restrict the applications of the mono-spectrum
marker. The disadvantage could be overcome with digital signal processors (DSPs). DSPs are used in mobile
devices for processing voice signals, and ﬁltering voice
signals is a major function. Such a function could be
used for extracting the mono-spectrum marker. On theother hand, GPUs are also mounted on into recent mobile devices. They would be useful for extracting monospectrum markers.
References
1. Aruga, H., Toyoura, M., Mao, X.: A trackable ar marker
in blurred or defocused images. In: Meeting on Image
Recognition and Understanding (MIRU), pp. 496–503(2011)
2. Asai, H., Oyamada, Y., Pilet, J., Saito, H.: Cepstral analysis based blind deconvolution for motion blur. In: IEEE
International Conference on Image Processing (ICIP)(2010)
3. Bay, H., Ess, A., Tuytelaars, T., Gool, L.V.: Surf:
Speeded up robust features. Computer Vision and Image
Understanding 110(3), 346–359 (2008)
4. Bichlmeier, C., Wimmer, F., Heining, S., Navab, N.: Contextual anatomic mimesis: Hybrid in-situ visualization
method for improving multi-sensory depth perception in
medical augmented reality. In: International Symposiumon Mixed and Augmented Reality (ISMAR), pp. 129–138(2007)
5. Dixon, J.D., Mortimer, B.: Permutation Groups.
Springer-Verlag (1996)
6. Fiala, M.: Artag, a fiducial marker system using digitaltechniques. In: IEEE Conference on Computer Vision
and Pattern Recognition (CVPR), pp. 590–596 (2005)
7. Ito, E., Okatani, T., Deguchi, K.: Robust planar target
tracking and pose estimation from a single concavity. In:
International Symposium on Mixed and Augmented Reality (ISMAR) (2011)

8

Masahiro Toyoura et al.

8. Kanatani, K., Sugaya, Y.: Performance evaluation of iterative geometric fitting algorithms. Computational Statistics and Data Analysis 52, 1208–1222 (2007).
9. Kato, H., Billinghurst, M.: Marker tracking and hmd calibration for a video-based augmented reality conferencing
system. In: International Workshop on Augmented Reality, pp. 85–94 (1999)
10. Klein, G., Murray, D.: Parallel tracking and mapping forsmall ar workspaces. In: International Symposium on
Mixed and Augmented Reality (ISMAR) (2007)
11. Lowe, D.G.: Distinctive image features from scaleinvariant keypoints. International Journal of Computer
Vision 60(2), 91–110 (2004)
12. Mohan, A., Woo, G., Hiura, S., Smithwick, Q., Raskar,
R.: Bokode: Imperceptible visual tags for camera based
interaction from a distance. ACM Transactions on
Graphics 28(3),98 (2009).
13. Newcombe, R.A., Lovegrove, S., Davison, A.J.: Dtam:
Dense tracking and mapping in real-time. In: International Conference on Computer Vision (ICCV), pp. 2320–
2327 (2011)
14. Okumura, B., Kanbara, M., Yokoya, N.: Augmented reality based on estimation of defocusing and motion blurring from captured images. In: International Symposiumon Mixed and Augmented Reality (ISMAR), pp. 219–225(2006)
15. Owen, C., Xiao, F., Middlin, P.: What is the best fiducial?
In: IEEE International Workshop on Augmented Reality
Toolkit (2002)
16. Park, Y., Lepetit, V., Woo, W.: Esm-blur: Handling andrendering blur in 3d tracking and augmentation. In: International Symposium on Mixed and Augmented Reality (ISMAR), pp. 163–166 (2009)17. Shan, Q., Jia, J., Agarwala, A.: High-quality motion deblurring from a single image. ACM Transactions on
Graphics 27(3),73 (2008)
18. Tateno, K., Kitahara, I., Ohta, Y.: A nested marker foraugmented reality. In: IEEE Virtual Reality, pp. 259–262(2007)
19. Uchiyama, H., Saito, H.: Random dot markers. In: IEEE
Virtual Reality (IEEE VR) (2011)
20. Uematsu, Y., Saito, H.: Improvement of accuracy for 2dmarker-based tracking using particle filter. In: International Conference on Artificial Reality and Telexistence(ICAT), pp. 183–189 (2007)
21. Xu, A., Dudek, G.: Fourier tag: A smoothly degradable
fiducial marker system with configurable payload capacity. In: Canadian Conference on Computer and Robot
Vision (CRV), pp. 40–47 (2011)

Masahiro Toyoura received
the B.Sc. degree in Engineering, M.Sc. and Ph.D. degrees in Informatics from Kyoto University in 2003, 2005
and 2008 respectively. He iscurrently an Assistant Professor at Interdisciplinary Graduate School of Medical andEngineering, University of Yamanashi, Japan. His researchinterests are augmented reality, computer and human vision. He is a member of ACM

SIGGRAPH and IEEE Computer Society.

Haruhito Aruga received
the B.Sc. and M.Sc. degrees
in engineering from University
of Yamanashi, Japan. His research interests include augmented reality and computer
vision.

Matthew Turk is a professorof Computer Science and former chair of Media Arts andTechnology at UC Santa Barbara, where he co-directs the
UCSB Four Eyes Lab. He received a PhD from the Massachusetts Institute of Technology in 1991 and has worked
at Martin Marietta Aerospace,
LIFIA/ENSIMAG (Grenoble,
France), Teleos Research, andMicrosoft Research. He is on
the editorial board of the
ACM Transactions on Intelligent Interactive Systems andthe Journal of Image and Vision Computing. He is a Fellow of the IEEE and the recipientof the 2011-2012 Fulbright-Nokia Distinguished Chair in Information and Communications Technologies.

Xiaoyang Mao received her
B.Sc. in Computer Sciencefrom Fudan University, China,
M.Sc. and Ph.D. in Computer Science from University of Tokyo. She is currently a Professor at Interdisciplinary Graduate School of
Medical and Engineering, University of Yamanashi, Japan.
Her research interests include
texture synthesis, non-photorealistic rendering and theirapplication to scientific visualization. She is a member of
ACM SIGRRAPH and IEEE
Computer Society.

Mono-spectrum Marker: An AR Marker Robust to Image Blur and Defocus(a) Original image(b) σt = 4.0(c) σt = 7.09(d) σx = 4.0(e) σx = 7.0

140

140

120

120

120

100

100

100

100

80
60

80
60

60

40

40

20

20

20

0
0

100

200
300
Frame

400

500

100

200
300
Frame

400

500

80
60
40
20

0
0

0
0

100

200
300
Frame

400

500

140

140

120

120

120

100

100

100

100

80
60

80
60

80
60

40

40

40

20

20

20

0

0
0

100

200
300
Frame

400

500

100

200
300
Frame

400

500

(b) σt = 7.0

0

100

200
300
Frame

400

500

0

100

200
300
Frame

400

500

80
60
40
20

0
0

(a) σt = 4.0

Error (mm)

140

120

Error (mm)140

Error (mm)Error (mm)

80

40

0

Error (mm)

140

120

Error (mm)140

Error (mm)Error (mm)Fig. 5 Results of marker detection on blurred and defocused images. (Top-left) Original image of 67th frame. (Top) Blurred((b) and (c)) or defocused ((d) and (e)) images. (Middle) Teapots placed on the marker positions detected as an ARToolKit
marker. The marker was not detected when images were blurred and defocused, and detected on farther positions when images
were defocused. (Bottom) Teapots placed on the marker positions detected as a mono-spectrum marker in the blurring image.
The marker positions were correctly estimated. The marker could not be detected in severely defocused image.

0
0

100

200
300
Frame

400

500

(c) σx = 4.0(d) σx = 7.0

Fig. 6 Results of marker detection. (Top) Average errors of four corner positions of detected markers as ARToolKit markers.
(Bottom) Errors as mono-spectrum marker. The x-axis is the number of frames, and the y-axis is the error.

Resolution of images
640 x 480
320 x 240
Ave.
19.5

Var.
231.5

Time
9.6

Ave.
20.4

Var.
243.6

Time
5.0

20.1

7.3

8.7

10.0

2.9

4.3

5.4

48.8

61.0

6.4

59.5

16.4

3.6

3.4

60.3

3.3

4.0

16.4

Processing time
[ms/frame]

ARToolKit marker(Blur, σt =7.0)
ARToolKit marker(Defocus, σx=7.0)
Mono-spectrum marker(Blur, σt =7.0)
Mono-spectrum marker(Defocus, σx=7.0)

Resolution: 640x480
Resolution: 320x240
100
53.8

58.3

64.0

16.4

17.4

18.7

50

0

(a) Errors and processing times (ms/frame) of marker detection.
(Not including frames in which markers could not be detected.)
Fig. 7 Errors and processing time.

1
2
3
Number of band-pass filters(b) Processing time versus number of band-pass filters.

10Masahiro Toyoura et al.

Fig. 8 Marker detection in various illumination and various scales, indoor and outdoor. The images were captured by Logicool
Qcam Pro 9000 with auto white balance, auto exposure and autofocus functions. The teapots are placed at the center of thedetected markers. Even though several images are blurred or defocused, the markers could be robustly detected. Especially,
the images in the third column include cluttered backgrounds, which consist of the same frequency components with monospectrum dots. The mono-spectrum marker could be correctly detected even in the images.