2021 年 度

修士論文概要

主査

舟橋 健司

副査

黒柳 奨

研究室

舟橋研究室

入学年度

2020 年度

学籍番号32414034

氏名

Shengyu Yan

論文題目

AR Paper Document Zoom Glasses System for Presbyopic People
老視者のための実際の紙ドキュメントが拡大されたように見える AR 拡大眼鏡システム

1Introduction
E-books are easy to be zoom-in, but older people

are prone to presbyopia and they are more inclined
to read printed books. Although reading glasses areoften used when the people suﬀering from presbyopia
to read paper document, it is necessary to repeatedly
remove and put on the glasses, when they are for example taking notes and other operations on a desk. It

Figure 1: (a) See-through HMD glasses, (b) Appear-

is diﬃcult to cope with such complex situations evenance of an experiment.

if using multifocal glasses. By the way, the automatic
display zoom system of a smartphone and a tablet for

user’s ﬁeld of view through the AR glasses (Fig. 1(b)),

presbyopic people has been proposed in our laboratory

and a user-adjustable magniﬁcation is provided.

[1]. In this paper, I propose an AR paper document
zoom system using see-through HMD glasses. Presbyopic people can read enlarged paper documents easily
through the glasses.

2

System over view

3

Prototype system
First a camera on the HMD obtains the user’s ﬁeldof view image. Based on the assumption mentioned
above, several scans in diﬀerent directions are performed to obtain the four vertices of the target area.

In order to realize AR paper document zoom-in, aThe highest and lowest vertices in the target area arecamera is of course needed to shoot at paper docu-identiﬁed, and a slope between the two points is cal-ment. The image information in the reader’s ﬁeld ofculated (Fig. 2(a)). The remaining two vertices areview is needed to be shot in real time ﬁrst. The camera

scanned from side edges to center (Fig. 2(b)). Whenis mounted on a HMD glasses (Fig. 1(a)). Then thea user holds a document, the user’s hand may cover

paper document area is extracted from camera image

the target document area, in which case occlusion oc-to prepare for only document area zoom-in. It is notcurs in the area. In particular, when the hand covers

a simple task to extract the document area from gen-the vertices of the target document area, the accu-

eral scene image. A paper may bend, and the edges ofracy of determining the target area will be reduced.

it may not become smooth straight lines. The paper

To solve this problem, the target area is scaned mul-material is able to be similar color of background. The

tiple times in four parallel directions, up, down, left

background situation will also become complex, andand right, to obtain the coordinates of the boundary

sometimes there may be a frame image like a paper

points of the target area (Fig. 3(a)). Based on these

document. In order to validating my idea of AR zoom

coordinates, the correct slope is calculated and ﬁlteredglasses, it is assumed for a prototype system that theto construct a linear equation. The intersection points

paper document is a clear white rectangle in dark color
background that is not white, and does not bend. Theof each line are calculated by the equation to obtainreader’s hands or other things do not cover the corner

matter how the area is rotated or how many verticesof it. However the paper can be rotated in 3D space.

are missing, these covered vertices can be correctly

Finally the clipped document area is overlaid in theidentiﬁed, and at the same time, slight variations of

the coordinates of the vertices of the target area. No

Figure 2: (a) Determine the highest and lowest featurepoints, (b) Determine remaining two feature points.

Figure 4: (a) Field of view from camera, (b) Zoom-in
view through glassses, (c) Enlarged image for display,(d) Illustrative composite image as ﬁeld of view.
read the text on the white document, and take somenotes about it in the meantime. Next, I asked subFigure 3: (a) Determination the missing vertex, (b)jects to wear the HMD, and perform the same work

Determination with lighting variation.

again. When a subject click the zoom button of thesystem, it starts scanning the camera image. The pa-lighting do not aﬀect the results (Fig. 3(b)). Although

per area is automatically recognized, and the enlarged

the user sees the dark color background and white pa-

area is overlaid on the HMD (Fig. 4(b),(d)). Subjects

per that are originally included in the ﬁeld of view(Fig. 4(a)), they also see the enlarged part of the pa-

can change magniﬁcation of zoom manually. After allthese, I used a questionnaire to obtain the reading

per extracted from the camera image (Fig. 4(b)). It isexperience and feelings of all the experimental sub-diﬃcult to take an AR seen photo through an HMD,

jects on whether they were equipped with HMD oran illustrative composite image is shown. The point ofnot. Subjects indicated that it is easier to operate

the document as the center of the view stays after en-

than adjusting the glasses and moving head or thelarged (Fig. 4(c)). An user sees AR overlaid view likedocument to focus, and to ﬁnd the appropriate mag-the illustrative composite image as the ﬁeld of view

niﬁcation using touchpad. They also mentioned that(Fig. 4(d)).

this system can make it more convenient to take notesor other hand’s tasks. In the future, target area recognition should be generalized for, i.e. any background

4Experiment and conclusion

or any document, by applying CV technology. It isimportant how the technique compares to bifocal or

An HMD glasses device; EPSON BT-300 was used
to build the prototype system (Fig. 1(a)). I selecteda few readers whose age around 50 as I experimentalsubjects. Although its focal length was not written

progressive lenses. I also would like to detect the distance between the eyes and the paper document, anddetermine whether enlarge operation is needed automatically [1].

on the manual, I found this HMD glasses could ﬁt topresbyopic people whose focus length of 300mm and500mm respectively in the pre-experiment. In the ex-References

periment, First I needed subjects to face a black plate[1] Huiyi Fang and Kenji Funahashi，“Automatic dis-with white document (Fig. 1(b)), and move to a dis-play zoom for people suﬀering from presbyopia”,

tance that is diﬃcult to see. Then I asked them to

SIGGRAPH, 2018.