International Conference on Artificial Reality and TelexistenceEurographics Symposium on Virtual Environments (2015)
M. Imura, P. Figueroa, and B. Mohler (Editors)

A Study for Vision Based Data Glovewith Back Image of Hand
Yutaro Mori , Kazuki Kawashima† , Yu Yoshida‡ , Masahiro Okada§ , Kenji Funahashi
Nagoya Institute of Technology, Gokiso-cho, Showa-ku, Nagoya, Japan

Abstract
In this paper, we propose new Vision Based Data-Glove with back image of the hand. Although we have proposed
Vision Based Data-Glove with palm image of a hand in our previous research, some fingers often are hidden overthe palm in back image of the hand. It can estimate hidden finger joint angles by using aspect ratio of hand image,hand area in 2D-image and length of convex hull outline.
Categories and Subject Descriptors (according to ACM CCS): Human-centered computing [Computer Graphics]:
Interaction paradigms—Virtual reality

1. Introduction
Researches to recognize hand posture from images is called
Vision Based Data Glove (VBDG). They are classified into
two groups. One uses database of hand postures [WP], but
the problem is that database is so large because of human
hand flexibility. Another detects fingertips from an input image [CFAR07]. However, it cannot estimate hand posturewhen a fingertip is hidden with self-occlusion. In our previous research we have proposed VBDG with palm imageof a hand [YFI12]. We assume that fingertips (and a tip ofthe thumb) can usually be detected from 2D image by using computer vision technology. Then each joint angle iscalculated by Inverse Kinematics (IK). When a fingertip isundetectable with self-occlusion, we have assumed that thefinger motion is continued for a while, then estimate the fingertip position and calculate angles. In this paper we propose
VBDG with back image of the hand. Although we also use
CV technique to detect fingertips, they often are hidden overthe palm. First the tip of the thumb is obtained by aspect ratio of hand image and thumb’s angles are obtained when it ishidden. Then each angle of fingers is obtained by hand areain 2D-image and length of convex hull outline.

† present affiliation is Genki CO., Ltd‡ present affiliation is IVIS, Inc.
§ present affiliation is Denso IT Solutions, Inc.
c The Eurographics Association 2015.

(a)

(b)

Figure 1: (a) Hand area and length of convex hull outline,(b) aspect ratio h/w.

2. VBDG System
When all fingertips are detected from 2D image, all joint angles are calculated by IK same as a palm image VBDG. Otherwise the system tries to calculate the total angle of thumb’s
joints. If a tip of the thumb is hidden, it obtains total angle of thumb’s joints first by using hand area in 2D-image,length of convex hull outline and aspect ratio of hand image (Fig. 1). Before using it, three dimensional table which
has the total angle of thumb’s joints as an element has been
made (Fig. 2). In addition each thumb’s joint angle is estimated from the total value, space did not permit us to insert

Y. Mori, K. Kawasima, Y. Yoshida, M. Okada & K. Funahashi. / A Study for Vision Based Data Glove with Back Image of Hand

Figure 2: Example of 3D table for total angle of thumb’sjoint.
Figure 4: Example of total bend rate.

Figure 3: Example of 3D table for total bend rate.

the details.
Then it obtains a total bend rate of four fingers using
hand area, length of convex hull outline and total angle ofthumb’s joints. Another three dimensional table which has
a total bend rate as an element has also been made (Fig. 3).
Total bend rate is represented from 0 to 400 points, and eachbend rate is represented from 0 to 100 percent for each finger (Fig. 4). Finally a part of detectable fingers is taken fromthe total bend rate, and other finger’s angles are estimatedlike dividing it for the fingers equally. We have to omit thedetails for want of space.
3. Experiment and Conclusion
We constructed an experiment system to confirm the effectiveness of the method described above. The pilot system
detects fingertips using color markers attached on the fingertips, and measures the hand position as 6 degrees of freedom
with an ARToolKit marker. Besides the image is assumed
as the just at the back of the hand. Fig. 5 shows an experimental result. The left is input image and CG generated withestimated angles. The right is palm side image and CG for
confirmation. The hand posture (finger joint angles) can be
estimated when the finger is hidden.

Figure 5: Experimental result.

In this paper, we proposed a new VBDG with back imageof the hand. It can estimate hidden finger joint angles by using aspect ratio of hand image, hand area in 2D-image andlength of convex hull outline. In the future, we would like toconsider the hand position and direction, it means, we should
treat multi dimensional table with pitch and roll of a hand.
References[CFAR07] C HEN W., F UJIKI R., A RITA D., R.TANIGUCHI:
Real-time 3d hand shape estimation using multiple cameras.
Proc. The 13th Japan-Korea Joint Workshop on Frontiers ofComputer Vision (2007), pp. 15–20.
[WP] WANG R. Y., P OPOVIC J.: Real-time hand-tracking with acolor glove. ACM Transaction on Graphics (SIGGRAPH 2009)
vol. 28, No. 3.
[YFI12] YAMAMOTO S., F UNAHASHI K., I WAHORI Y.: A study
for vision based data glove considering hidden fingertip with selfocclusion. Proc. SNPD2012 (Aug. 2012), pp. 315–320.

c The Eurographics Association 2015.