Manipulation of a VR object using user’s pre-motion
Shiori Mizuno1 , Asuka Komeda2 , Tomoko Yonezawa3 , Naoko Yoshii4 , Masami Takata4 , and Kazuki Joe4
1Graduate School of Humanities and Sciences, Nara Women’s University, Japan2
Department of science, Nara Women’s University, Japan
3
Faculty of Informatics, Kansai University, Japan
4
Graduate School of Humanities and Science, Nara Women’s University, Japan

1

Introduction

VR (Virtual Reality) is a technology to artificially generate a sense of reality by using computer graphics andacoustic techniques. Manipulation in VR has some problems: the necessity of pre-learning for manipulations, thediﬃculty of manipulation commands for some users, andlimited parts of available human’s motion. In this paper,
we propose a new method to intuitively and easily manipulate objects in VR. The purpose of the method is to givecommon users an instinctually simple interface to a VR
system with using their natural behaviors as their commands when they want to manipulate VR objects. To
capture the natural behaviors as the interface to VR, wepresent the definitions to classify the kinds of objects in
VR and the motions with which users may perform their
behaviors to face the classified objects. We focus the initial motions of users to face an object in VR. The initialmotions include ”hand pre-shaping” [1]. Hand pre-shaping
is known as human’s unconscious behavior to face a small
object to be grasped [2, 3]. We unconsciously prepare themotion in advance of the actual behavior to the facing
object according to the purpose of the behavior and theproperty of the object. In this paper, we call ”preparation of motion” pre-motion. The pre-motion diﬀers according to the purpose of behaviors and the property ofobjects. Given the definition of pre-motion, users’ will canbe guessed by the pre-motion. In this paper, the characteristics of pre-motions to a facing object with various
sizes, hardness and weights, are classified with the purpose and the property. To validate the pertinence of theclassifications, preliminary experiments are performed.
22.1

Classification definitions
Definition of object classification by feature

Aiming at the classification of VR object attributions
that trigger diﬀerent pre-motions, we investigate the classification of VR object features. We focus on the following
five features: shape, location, size, weight, and hardness.
2.2

Shape

We assume that human’s pre-motions are similar forthe target objects with a similar shape. Based on the assumption, we need to classify pre-motions by object shape.
Shapes are classified into several primitives. Human’s
holding motions are similar when the target objects areclassified in a shape with the same primitive.

2.2.1

Location

The direction of reaching out for an object varies according to the location where the object is placed. It turns outthat the location of an object plays an important role inunderstanding user’s will for the object in advance because
diﬀerent object locations cause the user to reach out forthe objects in diﬀerent directions.
2.2.2

Size

When we are going to hold an object, we unconsciously
decide whether we reach out one hand or both hands forthe object in advance. So we classify objects by size in two
categories: hand size objects to be grasped by one handand larger object to be held by both hands. It turns outthat a single hand is used for grasping an object with at
most hand-size while both hands are used for holding anobject with larger size than his/her hand. So the object
size dominates human’s motions of holding and grasping,
and the object size should be classified.
2.2.3

WeightLifting an object by hand, we judge whether we use one
hand or both hands by the physical appearance weight ofthe object. It turns out that the weight of the target object
aﬀects the motion of the user. So the weight of objects
should be classified in two categories: heavy (requires bothhands to hold) and light (requires one hand to grasp).
2.2.4

Hardness

When the size of the target objects is too large to grasp
by one hand, some objects are graspable by one hand. In
this case, the hardness of the target objects is diﬀerent:
soft and hard. The motion is possible when the target isenough soft that a part of the object can be deformed tobe grasped by one hand and the object is enough lightfor one-hand grasping. However, it turns out that someobjects are soft and deformable so that they are grasped by
one hand. So the hardness of objects should be classified
in two categories: hard (requires both hands to hold) andsoft (requires one hand to grasp).
2.3

Definition of motion classification

The motion of holding or grasping includes the premotion of reaching out one hand or both hands to thetarget object. When reaching out user’s hand(s), there are
two possible motions of touching and lifting. We believe

Fig. 2.
Reaching out both hands for (a)Touching(b)Pushing and (c)Lifting.

Fig. 1. The operation flow of the prototype system.

that the diﬀerence between the two motions is predictableby analyzing the shape of the fingers, the number of thehands, and the way of opening one’s hands regarding tothe object features.
2.3.1

The number of hands

When reaching out to an object, we judge whether we
use one hand or both hands regarding to the size, theweight, and the hardness of the object. In the case of
one hand, when the object size is small and the object
weight is light, we predict that the object is to be grasped
or touched. Even when the object is large, it could begrasped, provided that it is light and very soft. Otherwise
we predict it is to be touched. In the case of both hands,
we reach out to the object with opening the hands when
the object size is large. Even when the object is small, weuse both hands to hold the heavy object.
2.3.2

The shape of fingers

When the target object is enough small and light tobe grasped by one hand, the motion of reaching out may
be also a touching. To predict whether the motion is forgrasping or touching, we classify the shape of fingers. The
shapes of fingers, exactly saying the shapes formed by thethumb and the other fingers, are with sharp and right angles. Since the target object is hand-size, the distance
between the thumb and the other fingers is larger than theheight of the object. To grasp the object, since it is taken
between the thumb and other fingers with conforming thepalm, we understand this case represents a grasping motion. In short, the pre-motion for grasping includes a curve
formed by the fingers and a longer distance between thefingertips than the object height (or width).
2.3.3

Distance between two hands

In this paper, the horizontal, the vertical, and the depth
directions to the VR screen are defined as x-, y- and zaxis, respectively. The object width along the x-axis andy-axis are defined as sideways and longitudinal width, respectively. To hold an object by two hands, the distance
between two hands is prepared to be longer than the sideways or the longitudinal width of the object in advance ofactual holding. On the other hand, when the distance isshorter, the motion is predicted as touching. In this way,
the distance between two hands should be classified so thatthe change of holding and touching motion is predictable.
3Preliminary experiment

We perform preliminary experiments using a prototype
system provided that we focus a limited target with corre-sponding motions. The target object we use in this experiment is a cuboid. The preliminary experiment requires
motion capture ability. So we use Kinect as a sensor device to capture user’s motions. The operation flow of theprototype is shown as Fig.1. As the results of the experiment, we conclude that the prototype system based on thedefinition works well as. Figure 2 shows the user reaches
out her both hands. The distance between two hands isshorter than the sideway width of the cuboid, and theuser reaches out her hands toward a side of the cuboid.
It means that user’s request to VR objects is predictableand understandable based on the definition shown in subsection 2.2. Therefore we show that it is possible to takein human’s natural motion, as we here propose human’s
pre-motion, for manipulating VR objects.
4Conclusion

VR is one of the most eﬀective presentation methods
for giving any people various information that is diﬃcult to show such as world heritages or national treasures.
The most critical problem to use such VR is that typical VR systems require pre-defined control commands forpresented 3D objects. In short, it is diﬃcult for commonpeople, who are not familiar with VR operations, to manipulate 3D objects using such special commands in VR.
In this paper, we proposed the definition to classify therelation between user’s pre-motion and his/her target 3Dobject in VR. The purpose of the definition is to allow any
user to intuitively and naturally manipulate 3D objects inVR. In such a VR system, user can manipulate 3D object without learning control methods in advance. To getuser’s natural behavior as commands, definitions of classifications in user’s motions are constructed based on user’sinitial, here we say ”pre-”, motions. So we classify objects
and user’s motions to the objects. To validate the pertinence of the classification, a prototype system with Kinect
has been developed, and we performed some experimentswith the prototype. In this experiment, a cuboid is displayed in the prototype VR system to be manipulated by
an examinee. As the results, the examinee’s pre-motions,reaching out one/both hand(s), are recognized by the prototype based on the classification definitions. We conclude
that it is possible for ordinal people to intuitively and naturally manipulate 3D objects in VR. Our future work includes smarter classifications for more complicated objects
and various pre-motions.
References[1] Bard C., Troccaz J., Vercelli G., Shape Analysis and Hand
Preshaping for Grasping, IEEE, vol.15, pp.23-30, 1996.
[2] Jeannerod M., The timing of natural prehension movements, Journal of Motor Behavior, 16, pp235-254, 1984.
[3] Arbib M.A, Iberall T., and Lyons D. Coordinated control
programs for movements of the hand, Experimental brain
research, 10, pp.111-129, 1985.