Tablet VR-Learning System: Chemical Laboratory Experience System
Kyosuke Uchiyama
Nagoya Institute of Technology
Gokiso-cho, showa-ku,
Nagoya, Japanemail: uchikyon@center.nitech.ac.jp

Abstract—Education builds culture of human for many
years. The method of education has been developed fromoral communication and textbook to broadcasting and elearning. According to development, children have become
to be free for place, time and teacher who gives some advice.
On the other hand, the opportunities that children contact
with real things and experience is getting decrease. Generally
speaking, experience is important for the education such as
physical education, art and also science. In addition, it isusually difficult to have a good environment for experienceat a self learning situation and e-learning systems. In thispaper, we describe a chemical laboratory experience systemusing VR technology for primary school students, as a
VR-learning system. In addition, we confirm the usefulness
through experiments that several primary school students
use this system.
Keywords-Education culture; VR-learning; Chemical laboratory system; 3D tablet interface

I. I NTRODUCTION
Education is very important to build culture of human.
First we would like to look back upon bygone education.
When an institutional school did not exist, transmissionof information, in another word; education to children
depended on oral communication traditionally. However,it became to be necessary to socialize enormous numericalchildren systematically and uniformly in the industrial
society. Therefore cultural human being made the artificial
communal space such as a school, when there were someliteral people who used letter and character. They introduced a private elementary school at the Edo period forexample (figure 1) [1] [2] [3]. Then it had become possible
to transfer a large amount of the unified information byusing a textbook.
Times had changed after that, language education program was started to broadcast on radio at 1925 by Nippon
Housou Kyoukai (present NHK; Japan Broadcasting Corporation) (figure 2) [4] [5] [6]. Although people had beenable to see text and picture (include photo) printed on a
textbook, they became to be able to listen a good example
at home without a teacher (figure 3). Other education
programs were started also on TV, they could see, i.e.,
gesture and situation as a video. In addition a music tapeand video tape of them were started also to sale, students
were set free from not only place but also time.
The culture of education has changed recently, students
can browse electric textbook on PC, and of course watch
and listen sound and video. You usually call this system

Kenji Funahashi
Nagoya Institute of Technology
Gokiso-cho, Showa-ku,
Nagoya, Japanemail: kenji@nitech.ac.jp

as an e-learning [7] [8]. Some of these systems canmark answers for questions, also voice student vocalizes(figure 4). Students are still free from place, time andteacher who gives some advice, but we feel somethingis missing for education and we are not satisfied without
it.
While education is made efficiently, the opportunities
that children contact with real things decrease rapidly,
and the experiences in real life are getting evolved into
something artificial represented by TV, video and electronic device such as PC. Then the reality is going missing
from children’s environment. There is a surprising experiment [9]. The author gave a task to draw the picture of achicken for 153 children at Hiroshima Takehara Municipal
Yoshina junior high school in 1989, then 19 chidden drew
the chickens which had four feet (figure 5). Some of themdrew no-beak chicken and no-tail one. You can know that
there is a problem about education since lack of real life
experiences such as activities that bring them into contact
with nature.
Generally speaking, experience is important for theeducation such as physical education, art, music, craft,cooking and also science. It is usually difficult to havea good environment for experience at a self learning situation and e-learning systems. However the developmentof the education gives new various knowledge to a humanbeing, and it will lead to the development of culture.
In our laboratory, we focus on experience-type education system based on information technology, actually virtual reality technology, and develop a chemical laboratory

Figure 1.

Private school; Tera-koya [2]Figure 2.

Figure 3.

Radio text, English-conversation [4]Figure 4.

Self-study using e-learning system

Figure 5.

Chickens which have four feet [9]Illustration of education program on radio

experience system for primary school students [10] [11].
They can move actual beaker and flask on which position
sensors are attached, and see chemical reaction on visualmonitor, then confirm a procedure of experiment. We
are developing also VR cooking learning system (figure 6, 7) [12] [13].
It is necessary to represent liquid behavior to make VR
chemical laboratory experiment system. However hydrodynamics is too complex to solve in real time. There is an
SPH method which is a kind of particle method [14], andit makes possible to represent exact behavior of liquid forcomputer graphics and VR systems. However it still needs
enormous calculation, and high performance computer.
Therefore it is not suitable for our system that is designed
for using in educational institutions and households. Using
it at school, facilities have to prepare much number ofsystems. Students who have to stay at home and hospitalneeds low-cost and light-weight device. It means we haveto develop faster calculation method to represent liquidbehavior.
We have developed a particle and volume based liquid
model focused on real time processing and interaction.
In this model it is considers to divide liquid into two
conditions; one is the stay condition in a container, andanother is the free fall condition. The stay one is expressedwith particles which are not consider to be interfere
between each other. The fall one is managed based on
the volume in the container. This model realizes that user
can scoop, receive and spilling liquid with a container. Our
pilot system obtains position and orientation of a container
through a 3D motion sensor.
On the other hand, tablet devices become popular andare introduced to education platform as e-learning devices.
Also applying our system to more low-cost and lightweight device such as tablet, occasionally it is difficult
to propose 3D motion sensor for VR-learning system.
Then we try to develop the method to manipulate three
dimensional operation of liquid container using a touch
panel of tablet device. We propose new model that user
can rotate an object along latitude and longitude lines likethe earth after consideration of some typical methods. Thismodel can realize to rotate an object as expected without
discomfort.
As below, we describe the overview of a particle andvolume based liquid model and the chemical laboratory
experience system on PC in Section 2, and 3D rotation
model on tablet touch panel in Section 3. Then we show
experiments and conclude.
II. PARTICLE AND VOLUME BASED LIQUIDMODEL
A. Virtual liquid model
We consider that liquid is divided into two conditions inparticle and volume based liquid model we have proposed.
1) Free fall condition liquid: spills from a container anda tap2) Stay condition liquid: stays in a container such as a
cup

஼
ଵ

ଶ
ଷ

C

θ

φ

ସ

ψ

r

஼

C
஼

virtual container
Figure 8.

Figure 6.

Representation of virtual container

VR cooking learning system (1)

F
C
H

V

Figure 9.

Figure 7.

VR cooking learning system (2)The liquid of condition 1 is represented based onparticles. We do not consider interferences among particles
such as collisions and intermolecular forces between particles and the size of them. Each particle moves according
to only inertia and gravity. The particle coordinate Pi (t) attime t is calculated by the following equation (equation 1).
Pi (t) = 2Pi (t − ∆t) − Pi (t − 2∆t) + g∆t2

(1)

Where g is gravitational acceleration vector, and ∆t is therendering update interval.
The liquid of condition 2 is managed by the volume.
The volume of liquid in a container at time t is definedas Vt . It means, we can decide the height of liquidsurface where we know the volume and the attitude of
the container, of course without any wave and convection.
When one condition liquid transits each other, it is changed
to another condition with exchange rate N [number ofparticles / volume].
B. Virtual container model
In this model, we consider the sphere which containa container for simple interference detection. The centerof the sphere is C and its radius is r. In addition, we
define the container coordinate system (xc , yc , zc ) withthe origin C, and represent the attitude of the container

Liquid parameters for virtual container

with the intersection point between the sphere and theaxis yc (d1 in figure 8). First the container is rotatedby φ with the axis xc and point d1 moves to d2 . Then
it is rotated by θ with also the axis zc and point d2
moves to d3 . Its attitude is represented as the parameters(φ, θ). In addition its orientation is the rotation ψ withaxis yc where point d3 moves d4 . In a first pilot VR
system, these parameters are obtained using 3D position
sensor. Introducing tablet device to VR-learning system,
we consider to get these rotations through a touch panel
interface in the next section.
C. Liquid manipulation model with container
The liquid condition in a container changes from 2
to 1 when the container is tilted and liquid spills. First
we assume a convex container and some parameters aredefined as shown in figure 9. Where we assume the maximum volume of liquid which can exist in the container
as maximum capacity Vm . When the liquid volume V in
it exceeds Vm , liquid flows out at point F with exchange
rate N . The flow point F is defined as a relative vector
based on point C (equation 2). The liquid surface level
H is also defined as a relative hight based on point C(quation 3). And the equation 4 of the liquid volume V is
obtained from equation 3. Starting up a system, it makesa lookup table for these equations.
F = f (θ, φ, ψ)(2)

H = h(θ, φ, V )

(3)

z

x
Figure 10.

Grid for liquid surface

Figure 12.

Figure 11.

Figure 13.

Falling liquid into another container and wave

V = v(θ, φ, H)(4)

The liquid condition which falls changes from 1 to 2 whena container catches liquid. The particles which are caught
into it is added to the volume V of the container according
to exchange rate N . When a container skims liquid fromanother container, appropriate volume is transferred still
as condition 2. In addition a concave container like flask
is represented as some convex ones.
D. Representation of wave
The liquid condition 2 is stay condition as mentioned
above. However liquid is usually waving when a container
is moved and swung. Our idea was that we considered
steady state and that liquid surface as a basic surface first.
Then some effects are mapped on the basic surface. In
order to represent wave, liquid surface is devised into aorthogonal grid (figure 10), and wave is propagated by
two-dimensional wave equation as followed (figure 11).
∂2z
∂2z
∂2z
= v2 ( 2 + 2 )
2
∂t
∂x
∂y

(5)

Where v is the speed of wave propagation, and z is thehight of each grid point.
Furthermore, we considered vortex when liquid isstirred, i.e. by stick, and propose a concentric grid. We also
introduced the equation to represent an outline of vortex(figure 12) [10]. The outline is set at the basic surface
level, then concentric wave is mapped there to represent
vortex (figure 13).

Stirring liquid with red cube and vortex

Orthogonal grid and concentric grid

E. Representation of diffusion of color
We also considered about a diffusion of liquid color
using a grid (figure 14). Representing liquid surface more
realistic, the grid is configured with a few layers (figure 15). The diffusion happens according to the equation
below:∂P (x, y, z, t)∂ 2 P (x, y, z, t) ∂ 2 P (x, y, z, t)
= D(
+
∂t
∂x2
∂y 2
2
∂ P (x, y, z, t)
+
) (6)
∂z 2
P denotes the concentration of the liquid at a certain point,and D is the diffusion coefficient.
A similar method is used for the side surfaces. A slice
from the middle of the container facing the viewer but
parallel to the y axis is used for diffusion of color as shownin figure 16. The side projection is put onto the surfacesthat are visible from the viewpoint and that are facing
forward. The diffusion is then represented by applying a
texture on each polygon of the side surface (figure 17).
III. 3D ROTATIONS WITH TOUCH PANEL INTERFACE
A. Conversion from projected 2D circle to 3D sphericalcoordinate
Rotating an object in 3D space through 2D plane,
we first explain about the method to converse the 2D
coordinate on a plane to 3D coordinate on a sphere. We
consider that the sphere which contains a container in 3D
space is projected as a circle on a 2D screen which is atouch panel (figure 18). The locus touched with fingertip

Figure 14.

Figure 15.

Diffusion of color in liquid surface

Figure 16.

Mapping to the side surface

Figure 17.

Color diffusion on side surface

Multiple layers of grid to represent coloringfrom A to B is mapped to the locus from a to b on a
3D sphere (figure 19). Then we assume that there is ahemisphere on the screen, and its section matches to thecircle for easy understanding. The position touched on thetouch panel with fingertip is designated as the coordinate(X, Y ), and the radius of the circle is designated as R.
Then the height Z on a virtual hemisphere is calculated
by equation 7.
√
Z = R2 − X 2 − Y 2(7)
In other words, we obtain the 3D point (x, y, z) on thesphere which radius is r in 3D space from equation 8.
r
x
z
y
=
=
=
(8)
R
X
Z
Y
From this equation, we can get the three-dimensional
coordinate corresponding to the touch position.
B. Rotation through 2D interface
1) Rotation sphere which is fixed only center: There
are some methods to rotate an object in 3D space through2D interface, but each method has advantages and disadvantages. First we describe the rotation method where theobject center is fixed. As shown in figure 20(a), the 3D
points a and b is obtained from the 2D points A and B
on the touch panel. The rotation axis Q is defined as anormal line of the plane which contains the point a, b
and C. Then the object is rotated by λ on the axis Q.
However you feel uncomfortable when you move yourfingertip horizontally, because you usually image rotationof the earth and the top/bottom-touch rotation is different
from center one (figure 21).

2) Rotation sphere with axis parallel to screen: Next
we describe the rotation method where the rotation axis islimited parallel to screen. As shown in figure 20(b), the3D points a and b is also obtained mentioned above. The
rotation axis Q is defined as the line passing through thesphere center, and the projected line Q onto 2D screen
is perpendicular to the movement of finger from A to B.
Then the object is rotated by λ0 on the axis Q. Thus we
can solve the uncomfortable feeling of preceding model.
However the rotation on the vertical axis to the screen is
impossible in this model (figure 22), although preceding
one can realize it by drawing a path on a circumference.

b
λ

a

Q

C

C

C a

b

B

λ

Q

A

A

B

(a)

(b)

touch panel
Figure 18.

Projection to a touch panel

Figure 20.

Typical methods to rotate through 2D screen

Q

b

Q
λ

a

λ

B
A

(a)

(b)

touch panel
Figure 21.
Figure 19.

Different rotations between two parallel fingertip motions

Loci of circle and sphere

C. Rotation sphere using geographic coordinate system
Therefore we introduce a geographic (latitude and longitude) coordinate system, and propose a new interface
model. In this model the movement of the fingertip ona screen is decomposed into the latitude and longitude
components, and an object is rotated like a terrestrial
globe is rotated along longitude and latitude lines. A
point on the sphere is capable of multiple representations
as orthogonal coordinate system (x, y, z), a polar coordinate system (θ, φ) and geographic coordinate system(α(latitude), β(longitude)). First the 3D points a and b
is also obtained (figure 23), and the locus from a to b isdecomposed using point w. Where we define geographicalcoordinate of point a as (αa , βa ) and b as (αb , βb ), point
w is represented as (αb , βa ). Next the sphere is rotated on
the earth’s axis by latitude component, in other word, theamount of change in latitude parameter (βb − βa ). Finally
we consider the rotation such as the touched longitude
line will not move on appearance. The sphere is rotatedby (αb −αa ) on the axis Q which is the normal line of theplane contains the point a, w, C, in other word, contains
the longitude line βa (figure 24). Combining these rotation,
we determine the tile and the orientation (θ, φ, ψ) of acontainer after rotation based on the locus of the fingertip
on a touch panel.
D. Judgment on neighborhood of circumference
In our new model, you can rotate an object on theperpendicular axis to a screen when you trace over thecircumference. However it is difficult to trace it accurately
due to the conditions such as the sensitivity of a touch
panel and the thickness of a fingertip. We would like tointroduce the auxiliary circle that radius is larger than thecircle projected on the screen (figure 25(a)). When you
touch the area between the circle and the auxiliary one, theoperating point is assume as the intersection point between
the circle and the line from touched point to the circle
center (figure 25(b)). The rotation on the perpendicularaxis become easier.
IV. E XPERIMENTAL RESULT
We have already developed the first pilot system ofchemical laboratory experiment using the model mentioned above on desktop PC (figure 26). The main target
is primary school students. This system can help to understand chemical reaction and experiment procedure through

λ
C

஼

௔
஼

Q

a

Q

஼

Figure 22.

Rotation on vertical axis to screen
Figure 24.

Axis for latitude component movement

w
b

a

(a)
Figure 23.

Figure 25.

virtual experience. But we assume that the reaction andthe procedure are well known, and unknown experiments
are not considered. In this study, the virtual chemicallaboratory system using the interface mentioned above was
implemented on tablet device (figure 27). The specificationof tablet is the following;
Device: acer ICONIA TAB A500-10S16
CPU: Tegra2, 1GHz
• OS: Android Honeycomb 3.1••

In this experimental system, the operation of the mobile
container that an operator can move is realized as thefollowing.
The translation of 2DOF by normal drag operation
The translation which is perpendicular to the screenby the drag operations with two fingers• The rotation of the proposed model after toggled torotation model by tapping the container••

However we implement only some functions on tablet
because of the machine performance, i.e. except vortex
and side surface drawing.
Figure 28 and 29 show the appearances that primaryschool students operate chemical laboratory experiencesystem on tablet. The subjects are 9-year-old girl and6-year-old boy. They could move and rotate a container
through the touch panel as they expected only with simple
explanation. We got the following impressions about thissystem.
This was interesting, they had fun.
They felt that they really operated and mixed water.
• The girl understood the change well when mixing
two colors.
•
•

(b)

Decomposition of movement of fingertip
Auxiliary circle for perpendicular axis rotation

In addition, after our experiment they continued to use oursystem for more than an hour.
As a result, we found that our chemical laboratory
experience system on tablet is still useful to learn chemical
reactions, and our new interface to rotate an object in 3D
space through 2D screen is effective.
V. C ONCLUSION
In this paper we focused on education, and described
our VR-learning system; virtual chemical laboratory experience system. In order to make it become popular, wedeveloped our system on a tablet device that has become
popular and is introduced to education platform as ane-learning device. And we proposed the 3D operationinterface through 2D screen. Also it is confirmed that a
few primary school children could have a good experience
using the tablet VR system. This system is useful to obtain
experiences that is getting decrease when education isdeveloping. The following is considered as future works.
• Larger scale evaluation experiment in real field of
education• Develop a multi-touch 3D operation ingerface
We also would like to make VR-learning system popular to the education field, not only chemical laboratory
experience system, but also other VR contents such as
VR cooking learning system that is developed in ourlaboratory.
ACKNOWLEDGMENT
We thank our colleagues in our laboratory for useful
discussions. This work was supported in part by JSPS

Figure 26.

Chemical laboratory experience system on PC

Figure 27.

Chemical laboratory experience system on tablet

KAKENHI Grant Number 24501186.
R EFERENCES[1] Haruo Shirane, Tomi Suzuki, “Inventing the Classics:
Modernity, National Identity, and Japanese Literature”, Stanford University Press, 2002.
[2] Margaret Mehl, “Private Academies of Chinese Learningin Meiji Japan: The Decline and Transformation of the
Kanguku Juku”, Nordic Institute of Asian Studies, 2003.
[3] Benjamin Duke, “The History of Modern Japanese Education: Constructing the National School System, 1872-1890”,
Rutgers University Press, 2009.
[4] Nippon Housou Kyoukai, https://www.nhk-book.co.jp/
recommend/80-02anni/gogaku/, (in Japanese)[5] Robert L Hilliard, Michael C Keith, “The Broadcast Century
and Beyond: A Biography of American Broadcasting (5thedition)”, Focal Press, 2010.
[6] Karen Swan, Carla Meskill, Steven Demaio, “Social Learning from Broadcast Television (Media Education Culture
Technology)”, Hampton Pr. , 1998.
[7] D. Randy Garrison, “E-Learning in the 21st Century: A
Framework for Research and Practice”, Routledge, 2011.
[8] Rita De Cassia Veiga Marriott, Patricia Lupion Torres,“Handbook of Research on E-Learning Methodologies for
Language Acquisition“, Information Science Reference,
2008.

Figure 28.

Appearances of experiment (1)Figure 29.

Appearances of experiment (2)[9] Jun Kato, “Multimedia and Education”, Tamagawa University Press, 1990. (in Japanese)[10] Y. Natsume, A. Lindroos, H. Itoh and K. Funahashi，“TheVirtual Chemical Laboratory Using Particle and Volume
Based Liquid Model”, Proc. SCIS & ISIS 2010, pp.13541359, 2010.
[11] T. Tanabashi, H. Itoh, K. Funahashi, “Representation of
Wave Surface on Virtual Water Manipulation” Proc. SCIS
& ISIS 2008, pp.1460-1465, 2008.
[12] Y. Kurimoto, K. Funahashi, “The Collapse of Group of
Individual Bodies Using Transformation Surface for Virtual
Cooking System”, Proc. IWAIT 2013, pp.118-123, 2013.
[13] T. Ishihara, K. Funahashi, “Partial Sphere Container as Chinese Pan with Convex Bottom for Virtual Cooking System”,
Proc. IWAIT 2013, pp.957-962, 2013.
[14] Jun Chen, Kejian Yang, Yuan Yuan, “SPH-based visual
simulation of fluid” ,Proc. ICCSE, pp.690-693, 2009.
[15] S. Hasegawa, O. Sahara, A. Hasegawa, T. Tagawa, S. Ozaki,“Use of Tablet Terminals in Education : Introduction of iPadto Nagoya Bunri University”, Journal of Human Interface
Society, Vol.12, No.4, pp.245-252, 2010.