SharpView: Improved Clarity of Defocussed Content on Optical
See-Through Head-Mounted Displays
Kohei Oshima∗ Kenneth R Moser† Damien Constantine Rompapas∗
J Edward Swan II†
Sei Ikeda‡∗∗∗∗
Goshiro Yamamoto
Takafumi Taketomi
Christian Sandor
Hirokazu Kato∗ Interactive Media Design Laboratory
Nara Institute of Science and Technology(a)† Computer Science & Engineering‡ Mobile Computing Laboratory

Mississippi State University

Ritsumeikan University

(b)

(c)

(d)

(e)

Figure 1: The cause and effect of focus blur in Optical See-Through (OST) Head-Mounted Display (HMD) systems. (a) A user wearing the OST
HMD and related hardware used in our study. (b) Simpliﬁed schematic of an OST AR system. Blurring occurs when the virtual display screen
and real world imagery are viewed at unequal focal distances. (c), (d), (e): Views through an OST Augmented Reality system, where the realworld image (c) is in focus, causing the virtual image (d) to appear blurred; (e) an improved virtual image after application of SharpView.

A BSTRACTA common factor among current generation optical see-throughaugmented reality systems is ﬁxed focal distance to virtual content.
In this work, we investigate the issue of focus blur, in particular,
the blurring caused by simultaneously viewing virtual content andphysical objects in the environment at differing focal distances. We
examine the application of dynamic sharpening ﬁlters as a straightforward, system independent, means for mitigating this effect improving the clarity of defocused AR content. We assess the utility ofthis method, termed SharpView, by employing an adjustment experiment in which users actively apply varying amounts of sharpening
to reduce the perception of blur in AR content. Our experimental
results validate the ability of our SharpView model to improve thevisual clarity of focus blurred content, with optimal performance at
focal differences well suited for near ﬁeld AR applications.
Index Terms: H.5.1 [Information Interfaces and Presentation]:
Multimedia Information Systems—Artiﬁcial, augmented, and virtual realities; I.4.4 [Image Processing and Computer Vision]:
Restoration—Wiener ﬁltering
1I NTRODUCTION

Optical See-Through (OST) Head-Mounted Displays (HMDs) haveseen an increase in both popularity and accessibility with the release of several consumer level options, including Google Glassand Epson Moverio BT-200, and announced future offerings, suchas Microsoft’s HoloLens, on the horizon. The transparent display
technology used in these HMDs affords a unique experience, allowing the user to view on-screen computer generated (CG) content∗ e-mail: {oshima.kohei.of0, damien.rompapas.dk1, goshiro, takafumi-t,sandor, kato} @is.naist.jp† e-mail:moserk@acm.org, swan@acm.org‡ e-mail:ikeda.sei.jp@ieee.org

while maintaining a direct view of their environment, a property extremely well suited for AR systems. Unfortunately, the current generation of consumer-level OST HMDs are only capable of presenting CG content at a single ﬁxed focal distance. This inherent limitation becomes problematic as the user attempts to simultaneously
view the world and CG objects together, inducing focal rivalry as
the eye’s optical system must continuously adjust to accommodate
both the real and virtual items.
Existing techniques for correcting blur and distortion, producedby the lenses of OST HMD devices, use static correction schemes,
and therefore cannot adequately address the blur produced by focal
rivalry, which does not remain constant, but continuously changes
as the user ﬁxates on objects throughout the environment. Solutions
for OST focus blur require a dynamic approach, able to continually
adjust to match the changing focal demand of the user. The Point
Spread Function (PSF) [2], used to measure the focal ability of asystem, also has application for identifying the refractive power
limit of the eye’s internal lens [4]. In this work, we present anactive focus blur correction method capable of producing dynamic
image improvement by continuously updating a PSF model for theuser during system runtime. Our technique, designated SharpView,
is the ﬁrst methodology proposed for on-line processing of virtual
imagery to counteract focus blur in OST AR.
2

S HARP V IEW A LGORITHM

Consider the simple optical system, shown in Figure 2, used tomodel the eye and OST HMD screen. As the user’s gaze ﬁxatesat point M in the real environment, the light traveling from thispoint creates an image at point m on the eye’s imaging plane, after
passing through the lens. Similarly, light emitted from point M  on
the HMD screen is imaged at point m on the retina. The diffraction
pattern of the light is modeled by a PSF. This same procedure canbe analogously expressed, within the image processing domain, asa deconvolution operation between the image focused on the retina
and the PSF. Therefore, if a PSF is known in advance, it is possible
to restore a degraded image through deconvolution.
We generate our PSF approximation by modeling the intensity of
the light rays, emitted from the display screen, intersecting varying

(a)Figure 2: Optical system formed by the user’s eye and an OST HMD.
The imaging plane corresponds to the user’s retina and the lens aperture to the user’s pupil.

(b)

Figure 3: Views of evaluation task with real world reference images
placed in front of at 25 cm (a) and 500 cm (b) from the subjects’ eyes.
675650600200●points on the retina. The intensity distribution, p, can be represented by the following function.
1exp(−)
2πσ 22σ 2(1)In equation (1), σ represents the radius of the focal blur, while x
and y represent the pixel position of the emanating screen point.
Our simpliﬁed PSF model, therefore, requires only one parameter,
σ , to be determined at run time. σ itself can be derived from thebasic triangle ratio and lens equations as follows:
av 1 1σ= ( − )2 uu(2)●●σ°, ± 1 SEM

P(x, y) =x2 + y2

●
●
●
●

1.5

●

●

1.0
●

●

0.5

●
●
●
●

●

●

●
●
●
●

●
●

0.0
user sharpview

user sharpview

user sharpview

user sharpview

σ category

Once obtained, it is then necessary to scale from the eye’s image
plane back to the virtual display. If the radius of display blur isexpressed as σd , the ratio between the eye’s image plane and screen
is expressed as follows.

Figure 4: User-σ and SharpView-σ , expressed as visual angle,
grouped by focal disparity. The mean and standard error of the mean
(SEM) are shown in blue.

σ : σd = v : u

allowed to directly adjust the σ value of the corrective sharpening
ﬁlter until optimal clarity of the on-screen image is achieved.
Figure 4 provides the user selected σ as well as the predicted
σ values from our SharpView methodology, recorded for adjustment trials performed at four reference image focal distances. Ourresults show that the SharpView model is able to most accurately
produce sharpening correction for focus blur at a focus disparity
level of 650cm. For our system, which uses an Epson Moverio BT200 with a focal distance of 700cm, SharpView performs optimallywhen subjects view AR content in relation to real objects that are located at approximately 50cm, or arms length, away. This means thatapplications requiring the use of AR for near ﬁeld, or near vision,tasks, such as maintenance, 3D modeling, or medical procedures,will experience the largest beneﬁt from SharpView compensation.

(3)Here, σd is directly obtainable from equations (2) and (3),uaσd = (1 − )2u(4)

where a is pupil diameter, u distance between the eye and real world
gaze point, and u the distance from eye to HMD image plane.
3 S HARP V IEW IN ACTION
Common deconvolution operations include inverse ﬁltering,
Wiener ﬁltering, and the iterative Richardson-Lucy algorithm [1].
Difﬁculty in performing deconvolution with a PSF arises, however, due to inaccuracies in the diffraction pattern model. Okumura et al. [3] have utilized information from captured images of
2D markers to estimate the PSF of a camera based system. Unfortunately, the same principles are not applicable to the human eye,since only the user themselves have access to the perceived image.
Our SharpView algorithm overcomes these limitations by employing a simpliﬁed PSF model based on three user and system speciﬁcparameters: pupil diameter, distance to real world gaze point, andfocal distance of the HMD display screen.
We developed SharpView based on the premise that perceived
focus blur varies according to the focal disparity between the display and world. Based on this presumption, we implemented a user
study investigation utilizing a simple adjustment task to obtain measures for the amount of perceived focus blur and preferred level of
sharpening correction for subjects viewing real and virtual content
through an OST AR system. Focus blur is induced by making subjects ﬁxate their gaze upon real reference images, placed in frontof the display, at distances differing from the actual focal distanceof the HMD, Figure 3. A virtual image, identical to the real reference image, is then rendered to the display screen, and subjects are

ACKNOWLEDGEMENTS
This work is partially supported by a Bagley Graduate Assistant
Travel Grant awarded to Kenneth Moser and two grants fromthe Japan Society for the Promotion of Science (JSPS): Grant-inAid for Scientiﬁc Research (B) #15H02737 and Challenging Exploratory Research #15K12084.
R EFERENCES[1] R. C. Gonzalez and R. E. Woods. Instructor’s Manual for Digital ImageProcessing. Addison-Wesley, 1992.
[2] R. C. Jones. On the point and line spread functions of photographic
images. JOSA, 48(12):934–937, 1958.
[3] B. Okumura, M. Kanbara, and N. Yokoya. Augmented reality based onestimation of defocusing and motion blurring from captured images.
In IEEE International Symposium on Mixed and Augmented Reality(ISMAR), pages 219–225, 2006.
[4] J. Santamarı́a, P. Artal, and J. Bescós. Determination of the pointspread function of human eyes using a hybrid optical–digital method.
JOSA A, 4(6):1109–1114, 1987.