VR-Learning System: Virtual Cooking Training System
Kohei Sato and Kenji Funahashi
Dept. of Computer Science and Eng., Nagoya Institute of Technology,
Gokiso-cho, Showa-ku, Nagoya 466-8555 Japan
Corresponding Author: ksato@center.nitech.ac.jp
ABSTRACT
There are many kinds of education. The method of education has been developed
from oral communication and textbook to broadcasting into e-learning systemwith electric textbook on PC monitor. However it is usually diﬃcult to have agood environment for experience at a self-learning situation and e-leaning systems.
In this paper, we describe a cooking learning system using VR technology for
beginners, as a VR-learning system. Using our system you can learn cooking
anytime, anywhere, even if you have no time to learn cooking because of busyness.
In addition, we deﬁne the all objects such as rice and a piece of foods as a groupof individual bodies (GIB). We have proposed a manipulation model of the GIB
which calculates the GIB behavior according to transform surface, and enableshigh speed interactive operation.
Keyword: VR-learning, Virtual cooking system, Interactive manipulation, A
group of individual bodies, Transform surface
1. INTRODUCTION
There are many kinds of education, i.e., oral communication, self-education with
textbook, educational program broadcasted on radio/TV, and e-learning recently.
It is necessary to choose an appropriate method for your each purpose. The
textbook can make you free from place like school, and the educational program
liberate you also from time [NHK] [Hilliard] [Swan] (Fig.1, 2). The education
method has changed recently, you can browse electric textbook on PC monitor,and of course watch and listen sound and video. You usually call this system
as an e-learning [Garrison] [Marriott] (Fig.3). Some of systems can mark answer
for questions, also voice which student vocalizes. However it is usually diﬃcult tohave a good environment for experience at a self-education situation and e-learning
systems.
In our laboratory, we focus on experience-type learning system based on virtual
reality technology, and are developing a VR cooking learning system for beginners[Morii2010] [Morii2011]. Arrangements, planning and procedure are very important on cooking, for example, main dish which gets cold soon have to be cooked
last, stewed dish which takes some time have to be started to prepare, and an
expert usually make a salad and wash cookware already used while boiling something. Using our system you can learn time management on cooking anytime,Fig. 1: Radio text, English-conversation

Fig. 2: Education program on radio
anywhere, even if you must cook by yourself but you cannot go cooking school because of busyness. We are developing also VR chemical laboratory experience system (Fig.4) [Natsume][Tanabashi][Miyashita2011][Miyashita2012][Funahashi2000][Funahashi2001][Uchiyama].
There is another research about learning and training related to cooking [Kato].
They try to represent a process to grill meat. A part of our developing system isfocused on interaction of a gather of small bodies such as rice and a piece of foods,
i.e., cooking fried rice. Researches about lava and sand are related to our research[Stora][Onoue]. However they calculate the behavior of all solid materials one byone, so the amount of calculation takes much time. They cannot be applied for
an interactive system or cannot treat object globally.
In our idea, we treat the all objects which is in a cooking container such as frying
pan, in other words, a gather of small bodies as one object. We deﬁne the object asa group of individual bodies (GIB). We have proposed a manipulation model of the
GIB which calculates the GIB behavior according to the forces eﬀected through acontainer, and enables high speed interactive operation [Ishihara][Kurimoto]. In
this model, a GIB is not represented with a gather of individual particles but aheight ﬁeld which is deﬁned on the bottom surface of a container. We enable torepresent natural behavior that user expect (Fig.5).
In this paper, we describe GIB manipulation model with spatula-formed cook-Fig. 3: Self-study using e-learning systemware which user can operate freely on VR cooking system, especially pressing andscooping operations. This manipulation has diﬃcult problems and we assume thatthe operating surface of a spatula is vertical or horizontal. In chapter 2 we explain
the manipulation model of a GIB that we have proposed. In chapter 3 we propose
GIB operations with spatula-formed cookware. Then chapter 4 shows experiments
and results of the proposed system.

Fig. 4: Chemical lab experience system

2. MANIPULATION MODEL OF GIB
2-1. Container and transform surface
In this model, we treat the GIB existing in one cooking container as one operation
object even if it is divided into multiple in an appearance. First we deﬁne thecooking container as below.
• Bottom of container: Plane ﬁgure of convex polygon• Side of container: Vertical to the bottom

Fig. 5: VR cooking learning system
In addition, a GIB is deﬁned as a height ﬁeld which is ﬁxed on the container
bottom (Fig.6). When the volume of a part of GIB on the grid (x, y) is deﬁned as
V (x, y), the GIB volume Vc on the container is calculated as below.

Vc =

∑

V (x, y)

(1)

‫ݖ‬
‫ݕ‬
‫ݔ‬
Fig. 6: GIB on a Container
The behavior of the GIB is represented by varying the height of each grid in eachtime appropriately. In particular, we represent the variation of the GIB behavior
in a curved surface which is called Transformation Surface (TS). For example,
when a GIB exists in the container (Fig.7-1), it is expected to slide down to thelower part of the container according to gravity. Then, the TS is applied aroundthe GIB (Fig. 7-2). As a result the GIB is moved (Fig.7-3).
Actually GIB behavior is complex, so it is diﬃcult to calculate the shape of TS.
Thus, we use an alternative process. We think a GIB on the bottom receives
constant forces from gravity, friction and reaction form the container. Therefore,
we use a TS of a half of elliptic cylinder shape to simplify calculation (Fig.8).

1

2

3

increasedecrease

Apply TSt‫ ݐ‬+ ∆‫ݐ‬

Fig. 7: Vertical section of GIB in cylinder container
Parameters of the TS are deﬁned as below (Fig.9). The size of the TS is determinedfrom the semi major axis of elliptic a and the semi minor axis of elliptic b. The
place and the direction of the TS depend on the center of TS oD and the center
axis LD . The length of a is calculated from the volume of the GIB Vc and reaction
force form the side FN . The length of b is calculated from Vc , the appropriate
force that the GIB is received Fb , a static friction force from the bottom µ, anda dynamic one µ′ . The coordinate of oD is determined from a, the center of the
GIB Gc , and the direction of Fb . These parameters are calculated as below.
a={T1 − |FN′ | (T1 ≥ |FN′ |)0b=(T1 < |FN′ |){√(|Fb | − µ′ )T2 Vc(|Fb | ≥ µ)(|Fb | < µ)0oD = Gc +

Fb
a
|Fb |

(2)

(3)

(4)

The TS can represent a diﬀerence of properties of the GIB such as viscosity bychanging constants T1 , T2 . The axis LD is perpendicular to the direction of Fb ,passing on oD . The length of rD is appropriate constant which is longer than theradius of container bottom.
Using the TS, a GIB on a tilted container changes its shape to represent motion(Fig.10). To increase volume of GIB at lower part, a half of elliptic cylinder TS isapplied at the appropriate place (Fig.10-2). Then the volume of the TS is added
to the volume of the GIB (Fig.10-3). Later, the volume of the GIB is corrected(Fig.10-4). Finally, the GIB moves.
2-2. Shake GIB manipulation
A GIB in a container also should be transformed by shaking a container. However,
we have determined the position of a TS depending on the angle of the container,
in other words the gravity force. Consequently we regard the shape variation of a
GIB by shaking a container in the same light as one by tilting a container (Fig.11).
When a container is accelerated by the acceleration a, we consider that a GIB inFig. 8: TS applied on cylinder container

b

oD

a
LD

rD

Fig. 9: Half of elliptic cylinder TS

1

2

3

4

݃
‫ ݐ‬+△ ‫ݐ‬‫ݐ‬Fig. 10: Alternative process

a container receives a reaction force by the acceleration −a, and we apply a TS
according to the resultant force g ′ as relative gravity (Fig.12). Then we proceedshake manipulation according to the same process of tilting manipulation.

Fig. 11: Tilt operation and accelerated translation-
relative angle

′

Fig. 12: Relative angle for container acceleration

2-3. Toss GIB manipulation
In this model, we represent the toss GIB manipulation which we swing up a pan
and rise up pieces of ingredients in container into the air. This toss manipulation isused for the purposes of mixing ingredients, adjusting temperature, and extracting
water. However it is diﬃcult to deﬁne the height ﬁeld of GIB the bottom of apartial sphere container like a Chinese pan, so we introduce the container heightﬁeld to represent the shape of container (Fig.13). The height of the container fromthe height ﬁeld face hs (xi , yj ) on the grid (xi , yj ) is calculated by the radius of thepartial sphere Rs as below.
√
hs (xi , yj ) = Rs − Rs2 − (x2i + yi2 )(5)In the model using a partial sphere container, we apply a half of spheroid TS
to the appropriate position depending on the inﬂuence of gravity and the convex
bottom of the container. We represent a slide GIB manipulation by gravity on a

zs
ys

xs
ℎ (‫ݔ‬, ‫) ݕ‬

Fig. 13: Height ﬁeld for partial sphere container
partial sphere container to apply a TS similarly in order to maintain the rate of
the height of each grid.
When we shake a container aggressively such as a toss manipulation, a GIB receives force from a partial sphere container by changing the dynamic position. We
determine whether a GIB slides up or down on the container based on the directionof the force, and we represent an slide GIB manipulation to any direction applying
a TS as same as the slide manipulation by gravity. In addition, when a toss of a
GIB happens because a GIB slides up, it is necessary to convert a GIB expressedin a height ﬁeld into particles in the air. We call these particles as GIB particles.
The GIB particles are small solids gathered only in volume Vs , and they fall freely
according to gravity. We also determine the volume of conversion to particles byforce that a GIB receives from a container, though the parameters are determined
empirically. When the number of k GIB particles are generated and the numberof l GIB particles are received in a container at time t, the volume of the GIB in
the container V is calculated as below, and we represent a toss GIB manipulation.
V (t) = V (t − ∆t) − kVs + lVs(6)2-4. Probability particle and free fall particle
The minimum component of GIB in real world is a small solid, and it has a ﬁnite
size. However the GIB in a cooking container is represented with a height ﬁeld,
so it is not deﬁne the minimum component. Therefore, the minimum componentof GIB which is represented with a height ﬁeld is deﬁned as α, and we draw the
only GIB of the size that is higher than the minimum component α. If a height ofa grid f (xi , yj ) < α exists, the existence of GIB in the grid is judged statisticallyusing f (xi , yj ), and we draw the GIB according to the result. We deﬁne anotherheight ﬁeld to draw the GIB which existence is determined statistically. The valueof each gird in the drawing height ﬁeld f e (xi , yj ) is calculated as below according
to the height ﬁeld deﬁned in a cooking container.
• if f e (xi , yj ) < α{
f e (xi , yj ) =(P (xi , yj ))0(1 − P (xi , yj )) α(7)

• if f e (xi , yj ) ≥ αf e (xi , yj ) = f (xi , yj )(8)

Where P (xi , yj ) is the propability which the GIB exists in the grid (xi , yj ), and it
is calculated as below.
f (xi , yj )P (xi , yj ) =(9)αIn addition, we represent that GIB falls out from container with particle which
suppressed computational cost. Fig.14 shows the boundary if particles spill out
from container. The boundary A is for tilting operation, and the boundary B isfor pushing operation with cookware such as spatula to calculate simply. Besides,
θ is the angle that a piled up GIB keeps stability without collapsing voluntarily.
If a height of GIB exceeds either of the two boundaries, we represent the spill
manipulation by converting the particle based on the existence probability into
the free fall particle, and dropping into outside of container.

Boundary 

Boundary 

cooking container

Fig. 14: Spill boundary of GIB

2-5. GIB collapse with TS
In this section, we consider collapse of a part of a GIB, and propose the modelwhich we use the Gauss function to deﬁne a TS for collapse. Sectional views ofa typical image of a GIB before and after collapse are shown in Fig.15, 16. The
diﬀerence between the two Gaussian surface is deﬁned as the TS (Fig.17). A GIB
which is piled up continues to collapse until it gets stable. Therefore, we assume
the change of the piled up GIB during a certain time interval and use Gaussiansurface as the piled up GIB with smooth slope.

Fig. 15: Before collapse

Fig. 16: After collapse

Fig. 17: TS of collapse
First, we ﬁnd the areas where a part of a GIB may collapse and the center
point of that area. We compare the height of an observe gird and adjacent grids
to check the state of the piled up GIB locally. When an observe grid is thegrid (xi , yj ), we compare the height of the observe grid and four adjacent gridsh(xi−1 , yj ), h(xi+1 , yj ), h(xi , yj−1 ), h(xi , yj+1 ) (Fig.18, 19, 20). If the height of theobserve grid fulﬁlls both following conditions,h(xi , yj ) >h(xi−1 , yj ) + h(xi+1 , yj )
2

(10)

z

observe vertexhi −1 ix
i +1

Fig. 18: Observe vertex

i −1

i +1
j +1

i

y

j

j −1

z

x

observevertexFig. 19: Observe vertex and four adjacent grids vertex

h(xi , yj−1 ) + h(xi , yj+1 )(11)
2
we treat the grid(xi , yj ) as a center point of the part of the GIB may collapse(Fig.20-4). We keep the top of the grid (xi , yj ) as a candidate of a vertex for collapse. It is represented as a three-dimensional coordinate (xi , yj , h(xi , yj )). Moreover we consider a candidate of an area. It is a cone which has the vertex candidate
as its apes (Fig.21). We treat collapse in it as one object. An angle of the conic
slope of the area is an angle of repose which represents the angle that the GIB
piled up keeps stability.
In the above process, a lot of area candidates are detected. If we calculate collapseof a part of a GIB in each area, calculation time increases. Thus we reduce vertex
candidates using area candidates. For example, an area with a vertex candidate
P includes other vertex candidate Q (Fig.22). In Fig.22, we assume that Q is
included in the range of collapse centered at vertex P. We describe such collapse as
the collapse at P . Calculation of collapse at Q is not considered and Q is removed
from the vertex candidate list. For example, the top of the area candidate P andvertex candidates Q1 , Q2 are shown in Fig.21. In Fig.21, Q1 is removed from thevertex candidate. Candidate Q2 is kept as the vertex candidate list because it isoutside the cone. Vertex candidates are judged and reduced sequentially from thelength vertex. After this process, we treat candidates in the vertex candidate list
as a ﬁnal vertex candidates (Fig.23).
We judge whether a part of a GIB collapses using area candidates based on theﬁnal vertex candidates. If a cone-shape GIB, for example, has the apex which ish(xi , yj ) >

1

z

2

z

y

y

x

x
3

z

observevertex

4

y

z

y

x

x

Fig. 20: Four conditions with observe vertex and adjacent grids vertex

vertexcandidate ܳଵ
inside the conezvertex candidate ܲvertex candidateܳଶ outside the coneyarea candidatexθpart of container
Fig. 21: Area candidate and vertex candidates
the same height of P , and has smaller volume than the volume of the cone with the
apex P and the angle of repose θ, the angle of the GIB slope is larger than the θ.
Although a shape of a GIB is of course diﬀerent from a cone. we assume that thepart of the GIB in the area collapses at P when the GIB volume is smaller than
the reposed volume (Fig.24). The volume of the cone based on the area candidate
with the ﬁnal vertex candidate P is represented Vp , and the volume within thecone is represented VGIB (the volume inside the dashed line in Fig.24). Both Vp
and VGIB are calculated as following.
Vp =VGIB =∑(x,y)∈C(h(xp , yp ))3 tan2 θ
3

(12)

{
f (xi , yj ) : f (xi , yj ) < gen(xi , yj )gen(xi , yj ) : f (xi , yj ) ≥ gen(xi , yj )(13)

Height from generatrix gen(xi , yj ) is the height of the perpendicular line from thegeneratrix to a bottom at the grid (xi , yj ). Consequently R = VGIB /Vp < 1, theܲcandidatesܳ

removerange of collapse effectFig. 22: Reduce Candidateheight of final
vertex candidateܲheight of GIB‫ݖ‬height fromgeneratrix to bottom

ܸ௣
ߠ
‫ݕ‬

‫ݔ‬

Fig. 23: Height of GIB, Generatrix and Final Vertex Candidatepart of the GIB should collapse. We apply a TS and the part of the GIB collapses.
The central point G of the TS is matched with the ﬁnal vertex candidate P . We
can represent diﬀerence type of collapse with appropriate steps using the scaled
TS according to the size of the collapse area.

cone based onarea candidate

Vp

ܲ

V GIB

GIB
θ

θ

θ

ܴ=1

0 < R ≤1
Fig. 24: Stability Judged with Angle of Repose and GIB Volume

3. MANIPULATION WITH COOKWARE
3-1. Push GIB manipulation
We have enabled to operate a GIB shaking and tilting a cooking container, however
it is diﬃcult to operate it directly. This subsection explains the model to operate
GIB directly with rigid cookware. Although a frying pan is a kind of cookware,
we call this as cooking container, and we deﬁne cookware is a kind of spatula.

In this model, we assumed that an operating surface of spatula is rigid and composed of one or more rectangles (we call it the operating surface). And in this ﬁrst
step, we also assumed that the operating surface is always vertical (a handle isupside). Then, we regard three-dimensional convex hull which is movement trackof the operating surface (Fig.25), and we treat the interference of the convex hulland GIB as the one of the cookware and GIB.

Fig. 25: Three-dimensional convex hull
Cookware move in three-dimensional space, so the vertices constituting the convex
hull are represented by three-dimensional coordinate. When a height of GIB on
the grid (xi , yj ) is represented as h(xi , yj ), the point (xi , yj , h(xi , yj )) is deﬁned as
Pijh , and the point (xi , yj , 0) is deﬁned as Pijz identically. Then, if the following two
conditions are satisﬁed, we judge the GIB interferes with the convex hull Hch .
′′1. The convex hull Hch is projected to Hchor thographically, then Hchand the′grid Pij satisfy pij ⊂ Hch .

2. The intersection Pij of the convex hull Hch and the straight line lzh passing
through Pijz and Pijh exists, then the distance of Pijz and Pij is smaller thanthe value h(xi , yj ) of the height ﬁeld grid.
We apply the aﬃne transformation to the three vertices Ak , Bk and Ck of thetriangle Sk that constitute the convex hull so that the straight line lzh is vertical,
and we determine these as A′k , Bk′ and Ck′ (Fig.26). In addition, the straight line′transformed to be vertical is lzh. If the intersection of the triangle Sk′ and the′′′′zstraight line lzh
is Pijk, the coordinate of Pijk
is (xi , yj , zijk ), and the value of Pijk
is calculated as below.
′′z= Akz −
Pijk

′y
′y
′y
1
′x
′x
′x
′ z {nk (Pijk − Ak ) + nk (Pijk − Ak )}
nk

(14)

Besieds, n′k is the normal vector of the triangle Sk′ .
When the convex hull interferes the straight, the lower height on the grid (xi , yj )is hlow (xi , yj ), and the higher one is hhigh (xi , yj ) in order to distinguish two points(Fig.27).

′௛௜௝


௜௝௛

′
’



′
′




௭
′௜௝

௜௝௭
௭௛

′௭௛

Fig. 26: Aﬃne transformation
A part of the GIB that exists over the track of the operating surface is draggedby the friction (Fig.28). Then, if the operating surface interferes the grid (xi , yj ),we determine the subtracted value vijsub from the grid as below.

vijsub =



0






f (xi , yj ) − hlow (xi , yj )(f (xi , yj ) < hlow (xi , yj ))(hlow (xi , yj ) ≤ f (xi , yj )≤ hhigh (xi , yj ))f (xi , yj ) − (hhigh (xi , yj ) − hlow (xi , yj ))





f (xi , yj ) − hhigh (xi , yj )−(f (xi , yj ) > hhigh (xi .yj ))2(15)

Therefore, the total subtracted volume V sub is calculated using total number ofgrids N as follows.
V sub =

N∑

vijsub(16)

By subtracting the volume Vijsub from the grid (xi , yj ), we represent the GIB behavior which is transformed by the interference with the movement tracks of theoperating surface.
In addition, the positive TS which is applied by the interference with GIB andcookware is a quarter of elliptic cylinder, and the parameters which deﬁne the TS
is determined as below (Fig.29).
• Semi major axis of quarter elliptic cylinder a• Semi minor axis of quarter elliptic cylinder bℎ௛௜௚௛ (‫ݔ‬௜ , ‫ݕ‬௝ )ℎ௟௢௪ (‫ݔ‬௜ ,‫ݕ‬௝ )(‫ݔ‬௜ , ‫ݕ‬௝ )Fig. 27: Intersection of convex hull and straight line
• Width of quarter elliptic cylinder c• Position in space oC• Straight line perpendicular to long and short axis of ellipse LC
The position oC is determined by the reference point set on the operating surface.
The width of elliptic cylinder c is the one of the operating surface. The Straightline perpendicular to long and short axis of ellipse LC is parallel with the normalline of the operating surface, and it obtains the point oC . The semi major axis a
is set to be parallel with the normal line of the operating surface, and the semi
minor axis b is set to be parallel with it and perpendicular to the bottom of it.
The semi major axis a is determined from moving speed of cookware v, normalline of the operating surface n, and constant T3 as below.
a = T3 |v · n|

(17)

The value to add the grid by this quarter elliptic cylinder is calculated from thetotal subtracted volume V sub , so the semi minor axis b is a constant.
3-2. Press GIB manipulation
In our previous model, the operating surface of the cookware is always limited to
the vertical, it can be expressed as 4DOF operation. At the second step describedin this section, we assume that the operating surface of cookware is horizontal
for simplifying discussion focus. If a GIB is pressed with cookware, it is assumed
that the GIB around the operating surface is transformed. First of all, we judge
the interference between GIB and cookware, and subtract the interfered area of
GIB. Next, we represent the appearance of swelling of GIB around the operating

Subtracted volume

Convex hull

BeforeAfter

Fig. 28: Interference of convex hull and GIB

Operating surface

௖
௖











Fig. 29: Parameter of quarter elliptic cylindersurface when a GIB is pressed with cookware with half-torus TS which a torus isdivided into upper and lower. Furthermore, we represent more natural swellingof a GIB by changing the size and shape of half-torus according to the movement
speed of cookware and the area that is pressed with cookware.
The parameters of the TS are deﬁned as below (Fig.30).
• Large radius of torus R• Small radius of torus r• Position in container oT
A GIB receives a force from the operating surface by pressing the GIB with cookware. At this time, we assume that the force depends on the position of the center

‫ݎ‬

ܴ

‫்݋‬

Fig. 30: Parameters of half-torus
of gravity GT of the GIB which is pressed, and we deﬁne the position oT of the
TS is the center of gravity GT . The position of the center of gravity GT of GIB is
calculated as below(Fig.31, 32).
1 ∑ ′
GT =
mij
ST
N

(18)

Fig. 31: Pressing GIB manipulation
Where ST is the area of GIB which is pressed with cookware, N is the number′of gird that the operating surface interferes, and mij is the grid coordinate of theheight ﬁeld which is pressed. In addition, the large radius R of torus is calculated
as below by approximating it as the radius of the circle that is same as the area
ST .
√

ST
(19)
π
The volume of half torus is calculated by the volume which is subtracted by the
manipulation of pressing GIB. Therefore, the small radius r of torus is calculated
as below by using the volume V sub which is subtracted by the manipulation ofpressing GIB, and the large radius R.
R=

‫்ܩ‬

ܴ

்ܵ

Fig. 32: Pressing GIB manipulation

1
r=
π

√

V sub
R

(20)

3-3. Scoop GIB manipulation
A GIB is divided into the part that remains in container and the part that moves
on cookware when a GIB in container is scooped with cookware. Therefore, wedeﬁne the height ﬁeld on the operating surface of cookware. When the volume of
the GIB on the cookware grid (x′i , yj′ ) is deﬁned as V ′ (x′i , yj′ ), the GIB volume Vspon the cookware is calculated as below.
Vsp =

∑

V ′ (x′i , yj′ )(21)First of all, we consider the state that cookware interferes with a GIB by moving
cookware horizontally (Fig.33). If we pull the cookware in the upper or horizontal
direction, we move the part of GIB on the operating surface from the height ﬁeldof container to the height ﬁeld of cookware. The value vijmove of the grid which is
moved from the cooking container to the cookware is calculated as below by theusual convex hull.

vijmove =



0





0

(f (xi , yj ) < hlow (xi , yj ))(hlow (xi , yj ) ≤ f (xi , yj ) ≤ hhigh (xi , yj ))f (xi , yj ) − hhigh (xi , yj ) (f (xi , yj ) > hhigh (xi , yj ) and vxy < T4 )




high


 f (xi , yj ) − h (xi , yj ) (f (xi , yj ) > hhigh (xi , yj ) and vxy ≥ T4 )2

(22)

By the way, vxy is the movement velocity of a cookware in the plane xy, and T4 isconstant. Consequently, the total volume of a GIB which moves to the cookware
Vmove is calculated as following.
V move =∑vijmove(23)

In addition, a GIB moves onto a cookware, the value vijsub of grid is subtracted
by the interference with the operating surface of a GIB and the total subtracted
volume V sub is changed as below.
vijsub ={0(f (xi , yj ) < hlow (xi , yj ))f (xi , zj ) − hlow (xi , zj ) (f (xi , yj ) ≥ hlow (xi , yj ))∑V sub =
vijsub − V move

(24)
(25)

‫ݎ݋‬

Fig. 33: Scooping GIB manipulation
We suppose that a GIB maintains the shape because the operating surface of a
GIB is limited to horizontal, and we deﬁne the shape of the GIB on the operating
surface as a cone experimentally (Fig.34). Because the situation is complicated bycollapse of GIB both in container and on cookware. In this regard, we have beendeveloping separately. In the ﬁrst place, the center grid of the bottom of cone
Gs is calculated by weighted average where each weight is each grid height of theoperating surface respectively. We decide the radius of the bottom of cone r′ tocontact with the edge of the operating surface at least (Fig.35). In addition, theheight of cone fc to accord with the original volume Vsp is calculated as following.
3Vsp
(26)
r′2 π
If the angle between the bottom of cone and the bus ϕ is bigger than the angle ofrepose that we decided as θ previously, the radius of the bottom of cone r′ and theheight of cone fc are changed as below so that this angle ϕ is equal to the angleof repose θ.
fc =

√
3Vsp
πtanθ

(27)

fc = r′ · tanθ

(28)

r′ =

3

Besides, the center of the bottom of cone Gs is moved to the center of the operating
surface, and the shape and position of cone is decided as we mentioned before. At
this time, we limit the initial shape of GIB on the operating surface to a cone.
However, we enable the simple manipulation of scooping GIB by this method, andwe can sort out the problems to improve this model.

௖

ௌ

′

Fig. 34: Representation of scooping by cone TSௌ′Fig. 35: Representation of scooping by cone TS
4. EXPERIMENT AND RESULT
We constructed the experimental system based on the proposed model with C++
language and DirectX. The speciﬁcation of PC is the following; CPU: Intel Core
i5-2400, 3.10GHz. The appearance of the experimental system is shown Fig.36.
In this experimental system, we used the Wii remote released from Nintendo and
the PATRIOT released from PLHEMUS as input devices. The GIB is assumed
as fried rice, and the container is assumed as a frying pan. The GIB is drawn by
the texture mapping. First, the appearance of tilting, shaking, tossing, spilling,
and collapse GIB are shown in Fig.37, 38, 39, 40 and 41. Next, the appearanceof pushing, pressing, and scooping GIB with a cookware are shown in Fig.42, 43
and 44. These ﬁgures show that lots of kind GIB operations are realized.

Fig. 36: Appearance of experiment system

Fig. 37: Appearance of tilting container

Fig. 38: Appearance of shaking container

Fig. 39: Appearance of tossing GIB

Fig. 40: Appearance of spilling GIB

Fig. 41: Appearance of collapse of GIB

Fig. 42: Appearance of pushing GIB

Fig. 43: Appearance of pressing GIB

Fig. 44: Appearance of scooping GIB
In addition, we evaluated the relationship between grid size of height ﬁeld andprocessing speed in proposed model, actually without shake, toss and horizontalpress with spatula manipulation because of pilot system. The processing speed ofeach grid size is shown in TABLE 1. We assumed that the bottom of the cooking container is a circle with a diameter of 32.5cm, and the operating surface of
cookware is a square 100mm on a side. And we also adjusted the resolution of thecontainer grid and the cookware grid. According to the result, processing speedof the experiment is much faster than the speed that is required for interactive
manipulation (10∼12fps). However it is not excessive, because we have to implement toss and shake manipulation, and develop 6DOF spatula operation model.
We also would like to consider about washing, cutting and dishing up procedures
for a complete cooking learning system.
Next, several subjects used and evaluated the system. We set the gird size ofheight ﬁeld is 16.5mm×16.5mm in the experimental system. According to theevaluation, we got positive opinions. Moreover, we got some opinions that the
GIB behavior is natural and subjects felt to operate a cookware really.

TABLE 1: RESULT OF PERFORMANCE TEST
grid size [mm] processing speed [fps]16.5×16.5
12.5×12.5
9.5×9.5

721
496
286

5. CONCLUSION
We proposed an interactive manipulation model of a Group of Individual Bodies(GIB), and developed a cooking learning system as a VR-learning system. Thismodel can calculate GIB behavior with high processing speed with Transform
Surface (TS). Users can experience various manipulations, i.e., to move a groupof small ingredients in a cooking container, toss and spill it. We also proposed
the method how to calculate manipulations with spatula-formed cookware. Using
our system you can learn cooking anytime, anywhere, even if you have no timeto learn cooking because of busyness. As the experimental result shows, the proposed model can drive with high processing speed. This model can be applied tointeractive manipulation systems.
However the operating surface of a spatula is always limited to vertical or horizontal, because this spatula manipulation as 6DOF operation is still diﬃcult. Asa future work, we have to develop the method to calculate 6DOF spatula manipulation, and realize actual spatula operation. Furthermore we will develop themodels for other cooking processes such as washing and cutting ingredients anddishing them, then we would like to release the VR Cooking School system.
ACKNOWLEDGMENT
We thank our collegues in our laboratory for useful discussions. This work was
supported in part by JSPS KAKENHI Grant Number 24501186.
REFERENCES
Nippon Housou Kyoukai, https://www.nhk-book.co.jp/recommend/80-02anni/
gogaku/, (in Japanese)
Hilliard, R. L. and Keith, M. C. 2010. The Broadcast Century and Beyond: A
Biography of American Broadcasting (5th edition), Focal Press.
Swan, K., Meskill, C. and Demaio, S. 1998. Social Learning from Broadcast Television (Media Education Culture Technology), Hampton Pr.
Garrison, D. R. 2011. E-Learning in the 21st Century: A Framework for Researchand Practice, Routledge.
Marriott, R. D. C. V. and Torres, P. L. 2008. Handbook of Research on E-Learning
Methodologies for Language Acquisition, Information Science Reference.

Ishihara, T. and Funahashi, K., Partial Sphere Container as Chinese Pan with
Convex Bottom for Virtual Cooking System, Proceedings of IWAIT 2013, 957962, 2013.
Kurimoto, Y. and Funahashi, K., The Collapse of Group of Individual Bodies Using Transformation Surface for Virtual Cooking System, Proceedings of IWAIT2013, 118-123, 2013.
Natsume, Y., Lindroos, A., Itoh, H. and Funahashi, K., The Virtual Chemical
Laboratory Using Particle and Volume Based Liquid Model, Proceedings of SCIS
& ISIS 2010, 1354-1359, 2010.
Tanabashi, T., Itoh, H. and Funahashi, K., Representation of Wave Surface on
Virtual Water Manipulation, Proceedings of SCIS & ISIS 2008, pp.1460-1465,
2008.
Miyashita, S. and Funahashi, K., Envelopment Surface Rendering of Falling Water
for Particle and Volume Based Virtual Liquid Manipulation Model, Proceedingof NICOGRAPH International 2011, P02 (CD-ROM), 2011.
Miyashita, S. and Funahashi, K., Falling Water with Key Particle and Envelope
Surface for Virtual Liquid Manipulation Model, Proceeding of ACM VRST2012,
pp.197, 2012.
Funahashi, K. and Iwahori, Y., Manipulation of Liquid Using Cases in VirtualSpace, Proceeding of IEEE ROMAN2000, pp.368-373, 2000.
Funahashi, K. and Iwahori, Y., Virtual Liquid Manipulation Using General Shape
Vessel, Proceeding of IEEE VR2001, pp283-284, 2001.
Uchiyama, K. and Funahashi, K., Tablet VR-Learning System: Chemical Laboratory Experience System, Proceeding of SITIS2013 (Workshop on CIS), pp.416423, 2013.
Kato, F., Mitake,H. and Hasegawa,S., Interactive Cooking Simulator, In: Proceedngs of the 15th Virtual Reality Society of Japan Annual Conference (academic
lectures), 2D2-2 (DVD-ROM), 2010. (in Japanese)
Stora, D., Agliati, P. O., Cani, M. P., Neyret, F. and Gascuel, J.D. 1999. Animating
Lava Flows, Graphics Interface ’99, pp.203-210.
Onoue, K. and Nishita, T., Virtual Sandbox, In: Proceedings of the 11th PacificGraphics, pp.252-259, 2003.
Morii, A., Yamamoto, D. and Funahashi, K., Interactive Manipulation Model ofGroup of Individual Bodies for VR Cooking System, Proceedings of ICEC2010,
pp.484-486, 2010.

Morii, A., Kamigaito, S., Yamamoto, D. and Funahashi, K. 2011. Existence Probability Particle Based Interaction Model of Group of Individual Bodies for VR
Cooking System, Transactions of the Virtual Reality Society of Japan, Vol.16,
No.4, pp.539-549. (in Japanese)