Grasp Motion Planning for box opening task by multi-fingered hands
and arms
Tetsuyou Watanabe and Michael Beetz
Abstract— The aim of our project is to develop a robot tomanipulate an object in human environment. In this paper,
as a first step, we focus on opening paper box such as tea
box, and present a method to plan grasp motion by 2 arms
with multi-fingered hands. we propose a task priority based
scheme to plan grasping area consistent with whole steps ofthe given task procedure. Based on the grasping area and theconcept of preshape, we derive desired fingertip positions andhand base position and orientation for preshape. Based on thevector field approach, we propose a motion planning method
for the planned grasp by multi-fingered hands to avoid any
undesired collisions. This method can be applied to regrasping
and a motion in which collision is required.

I. I NTRODUCTION
Recently, there is much attention on robots executing everyday manipulation in human environment such as opening
box, opening package and folding clothes [1]. Multi-fingered
hand manipulation is necessary to execute such kind of tasks.
However, manipulation by multi-fingered hands and arms isvery complicated task for robot. In order to complete a task,
the robot has to create a procedure for the task, and to derive
motion trajectories for each step to complete the procedure.
Our project [2] proposed a method for creating the procedure.
First, a data-base for procedures is made inside internet space
so that the robot can search a procedure like human uses
search-engine. The robot picks up an appropriate procedureor task for the situation, and creates a procedure for the task.
This paper focuses on the task to open (deformable) paperbox (for example, tea box) as a first step, and develop agrasp motion planning method for the given task procedure.
Table I shows the scenario for the task and the computational
problems/issues to be resolved in this paper. Note that this
scenario is manually planned. But if embedding our approachto create the procedure, it will be planned by robot systemitself.
The main contributions of this paper are as follows: 1.
We propose a method to plan grasps by 2 arms with multifingered hands, consistent with whole steps of the given task
procedure. 2. We propose a method to plan a motion for theplanned grasp including grasp for movable object, in which
the robot can avoid any undesired collisions while making
desired collisions.
This work was partly supported by the German cluster of excellence
COTESYS (Cognition for Technical Systems), MEXT University Education
Internationalization Promotion Program and SCAT Research Grant.
T. Watanabe is with School of Mechanical Engineering, College of
Science and Engineering, Kanazawa University, Kanazawa, 920-1192, Japante-watanabe@ieee.org
M. Beetz is with Computer Science Department, Munich Universityof Technology, Boltzmannstrasse 3, D-85747 Garching Munich, Germany

beetz@in.tum.de(a) task 1(b) task 2(c) task 3(d) task 4(e) task 5
Fig. 1.

Overview of the task procedure

TABLE ITASK PROCEDURE AND THE PROBLEMS / ISSUES TO BE RESOLVED

Task procedure
Fig.1(a) The robot grasps the paper box
on the table by right hand.
Fig.1(b) The robot investigates its opening area by camera.
Fig.1(c) The robot regrasps the box byleft hand to open the box.
Fig.1(d) The robot crashes the box for
opening by right hand while
holding the box by left hand.
Fig.1(e) The robot opens the box byright hand while holding thebox by left hand.

Problems/Issues
Grasp and motion
planning
Algorithm for detecting opening area
Grasp and motion
planning to grasp
movable object
Grasp and motion
planning to grasp
movable object
Grasp and motion
planning for a motion with collision

When choosing the holding grasps the robot has to takethe intended manipulation grasps into account. For example,
when holding the box it must hold it such that the opening
is still possible, which means that the opening area must not
be covered by the holding hand. Also, the box face with the
opening area must face the robot.
We propose a priority-based decision scheme to determine
the grasping area at every step taking into account theprevious, current, and next steps/tasks. The priorities of thegrasping task then determine the priorities of the respective

grasping areas for both hands and the search area for camera
at each step, which we use to arrive at a globally consistent
manipulation plan.
We use the concept of preshape [3], [4] to make that thehand is in the appropriate configuration when approaching
the object, especially, for opening task requiring collision.
The effectiveness of using the preshape for this purpose wasshown by several researchers [3], [4].
In order to compute the preshape of the hand we takethe derived grasping area and construct a preshaping box(PB), which has the faces indicating fingertip positions forpreshape and the face indicating palm location. Since the PB
has all information for grasping (if roughly speaking), grasp
planning problem can be regarded as the problem of where
PB should be located for the respective tasks. Using thePB, we derive the desired preshaping positions and the palm
position and orientation for the target task. This problem isa nonlinear one. Thus, in order to reduce the computation
time, we propose a method to calculate them by a convex
problem obtained by applying a linearization to the problem.
Next, in order to make a motion to grasp the target object
with the desired state of fingers and hand base (palm), we
develop a method of motion planning which distinguishes
between desired (contacts) and undesired collisions andavoids the undesired collisions. Human environments oftenchange dynamically. To account for this dynamics, we use avector field [5] based motion planning strategy. In regrasping,
the object is grasped by a hand, and then the object is movedby the holding hand. When opening the box, a certain finger
must collide with the box while the other fingers must avoid
any collisions. We propose a method which can be applied
to a grasp for movable object and a motion with required
collision, based on the grouping method of the links and thebodies of the multi-fingered hand. This method can deal withthe change of finger configuration and new obstacles whichappear during manipulation.
A. Related works
As for detecting opening area, there is a texture-based
tracking method [6]. This method can detect the specified
texture area and track it if the texture is given.
As for multi-fingered hand, grasping position planninghave been researched by many researchers [7]–[9]. On theother hand, there are a few researches for grasp motion planning by arm with multi-fingered hand because it needs not
only grasp motion planning but also arm motion planning.
In addition, the volume and the workspace of fingers, andthe palm position must be considered. Miller et al. proposeda method for the grasp planning [10], using the grasping
simulator [11]. Harada et al. [12] proposed a method forhand and arm system to grasp an object based on convex
model. Morales et al. [13] proposed a method to plan agrasp using a database for grasping style calculated offline.
Berenson et al. [14] proposed a method to grasp a target
object in the environment where obstacles exist around theobject. However, they didn’t take into account task.

Fig. 2.

Initial state

As for plaanning for given task, Prats et al. [4] proposeda method of motion planning for hooking a door. But, theobtained result is limited. There are still many problems to
be resolved: for example, planning for other tasks, planningfor regrasping. Especially, there is no research about motion
planning for task procedure for 2 arms with multi-fingered
hands.
As for regrasping, there are many researches [15]–[17],but they took into account only finger(tip) motion.
II. P ROBLEM DEFINITION
We consider the following task performed by the robot
which has 2 arms with multi-fingered hands and camera.
1) The robot grasps a paper box on the table by righthand so that camera can see candidate faces including
opening areas (see Fig.1(a)).
2) The robot investigates the box by camera and detects
the opening area, rotating the box (see Fig.1(b)).
3) The robot grasps the box by the left hand to holdthe box avoiding the opening area so that at the next
step, the right hand can open/crash the box. After thegrasping, the robot removes the right hand from thebox (see Fig.1(c)).
4) The robot crashes (the opening area of) the box by theright hand to open the box while holding the box by
the left hand. After crashing, the right hand is removed
from the box (see Fig.1(d)).
5) The robot picks the opened area by the right hand toextend the opened area while holding the box by theleft hand. Then, the robot extends the opened area byrotating the right hand pinching the opened area (seeFig.1(e)).
Fig.2 shows the initial state. We use SCHUNK Anthropomorphic Hand (SAHhand) [18] which has 4 fingers which,respectively, correspond to thumb, index, middle and ring
fingers of human. We assume that the robot knows the
opening area is on one of the faces including the longest
edge, and around the middle point of the longest edge on theface including the opening area. It means the robot has tosearch the opening area. In addition, we assume that the robot
can use a tracking method such as a texture-based method[6], in order to search the opening area. The motion of thebox for searching opening area, and the motion of the handfor crashing and opening the box are assumed to be given.
This paper provides grasp and motion planning for completing this task.

Preshaping box

Faces for preshape

Fig. 3.

Face for palm

Grasping style and Preshaping box (PB)

III. G RASP PLANNING
Firstly, we make grasping area according to the task. Then,
we make preshaping box (PB) which roughly describes thepreshaping positions and palm location. Based on PB, wederive the desired fingertip positions and palm (hand base)position and orientation for preshape.
A. Preshape form
The shape of the object to be grasped is a box. Therefore,
the corresponding object model can be expressed by a box.
If the shape of the target object is not a box, we will
get the object model by splitting the object into several
parts, and calculating the minimum sized box for every partwhich can include the corresponding part like [12]. For thesimplicity, we use only one type of grasping style as shownin Fig.3. According to this grasping style, we define thepreshaping box (PB) as shown in Fig.3. PB has one facewhich expresses the preshaping position for thumb finger, its
antipodal face which expresses the preshaping positions forthe other fingers, and the face which expresses the locationof the palm. In this research, we use box shaped PB, but theshape of PB can change to a convex polyhedron according
to the grasp style. In addition, the following discussion is thecase when using only one type PB, but it can be extended
to the case when using several kinds of PB. However, detail
discussion is our future work.
B. Grasping region
To find the opening area, the robot must grasp the object
so that the camera can see all candidate areas for opening
area. If the robot regrasps at the next step, the robot shouldgrasp the object so that the robot can grasp the object by
the other hand. Then, different from the case when just only
grasping is required, we should find the grasping area consist
with not only the current task but also the next and previous
tasks. Then, we need consider whole tasks to determine thegrasping area for every task. Here, we propose a prioritybased decision scheme.
We set the priority of the task as follows.
Task 1: priority 1
Task 2: priority 1
Task 3: priority 2
Task 4: priority 1
Task 5: priority 1

Here, the number of task is associated with the numberof the task described at sectionII, and the priority is high if
the number of the priority is small. The priority of task 3
is only low since it is not specified, namely how to movefor regrasping is not given and we have to plan the motion.
Note that here, we manually determine the priority, but it ismuch better that the given procedure (created by data-base)includes the priority for every task.
Before setting the priority of grasping area and searcharea for camera, we consider the relationship between thegrasping areas for tasks: for example, whether or not thegrasping area for the current task is same as that for theprevious or the next task. Let Gri and Gli be, respectively,
the grasping areas for right and left hands at task i. Then,
according to the task procedure, the relationship can be
expressed by
Gr1 = Gr2 = Gr3 ,Gl3 = Gl4 = Gl5 .

(1)

Let pri (i = 1, · · · , 5) be the priority of (grasping area of)right hand for task i, let pli (i = 1, · · · , 5) be the priorityof (grasping area of) left hand for task i and let pci (i =1, · · · , 5) be the priority of (search area of) camera for task
i. If the hand uses the same grasping area as the previoustask, we set the priority of the hand equals the priority forthe hand for the previous task. Then, from the relationship(1), we get
pr1 = pr2 = pr3 ,pl3 = pl4 = pl5 .

(2)

According to the task priority and the relationship (2), we
set the priorities of grasping areas for hands and of searcharea for camera.
If the hand is the main hand to execute the specified task(searching opening area, crashing, and opening) at task i, we
set the priority of the hand for task i is the highest. We set
the priority so that at every task, the priorities for both hands
and camera can have different values from each other. If thepriorities for both hands are same (for example, consider
task 3), we set the priority of the hand which has the samegrasping area at the previous task is higher. Note that if we
can not get reasonable grasping areas (for example, nothing),
we change the priority or add a new task for regrasping, andrecalculate grasping areas. Then, we can get the following
priorities of the hands and the camera at every task.
pr1 = 2,
pr2 = 2,
pr3 = 2,
pr4 = 1,
pr5 = 1,

pl1 = ∗,
pl2 = ∗,
pl3 = 3,
pl4 = 3,
pl5 = 3,

pc1 = ∗
pc2 = 1
pc3 = ∗
pc4 = ∗
pc5 = ∗

(3)

Note that at task 1 and 2, the left hand is not used, and thenits priority does not need, and is set to ∗ (which means there
is no priority). When we don’t need to care about the tracking
area for camera to determine grasping area, the priority of
the camera is set to ∗.
According to the order of the priority, we determine
the grasping area, and search area for camera. Let Gci

C. Preshaping box
Grasping
area

Grasping
area(a) task 4(b) task 5
Grasping area for the task 4 and 5

Fig. 4.

Search area

Fig. 5.

Gr2

Area for searching opening area(i = 1, · · · , 5) be the search area for camera at task i.
First, we consider Gr4 , Gr5 and Gc2 , whose corresponding
priorities are 1. We define the component for the graspingarea, which consists of 3 faces which corresponds to the 2
faces for preshape/grasp and the face for palm of PB shownin Fig.3. We define that the grasping area is union of thecomponents. Since the ways of opening and crashing are
given, we determine Gr4 and Gr5 according to the specified
tasks as shown in Fig.4. From the assumption, the candidate
faces for the opening area are the faces including the longest
edge, and the opening area is around the middle point of thelongest edge. If splitting the object model into 3 parts asshown in Fig.5, the central part is the search area.
Next, we consider the grasping area Gr1 (= Gr2 = Gr3 )whose corresponding priority is 2. We calculate the unionarea of Gci and Gli whose priorities are higher than Gr1 at
tasks 1, 2, and 3. In this case, the union is Gc2 . Let Go be
the whole area of the box. Then, Gr1 is given byc
Gr1 = Go ∩ Gc2.

As it can be seen from Fig.5, Gr1 (= Gr2 = Gr3 ) consists of
2 separated parts.
Lastly we consider Gl3 (= Gl4 = Gl5 ). The corresponding
priority pl3 (= pl4 = pl5 ) is 3. We consider the other areas
for the same tasks, whose corresponding priorities are 1 or
2. The union of the areas is given by
Gru = Gr3 ∪ Gr4 ∪ Gr5 .
Then, Gl3 is given byc
Gl3 = Go ∩ Gru.

Based on the grasping area, we derive PB. Firstly, we
search all possible components (2 faces for preshape/grasp
and one face for palm) included in the grasping area. For
example, 4 candidate components are included in Gr1 . For
every possible component, we check whether the component
is available or not, based on the information about the object
state (whether a face is available for grasping or not: forexample, if the face contacts with the table, we can not use
the face), and the finger constraints (if the maximum distance
between thumb fingertip and the other fingertips is smaller
than the length between the 2 faces for preshape/grasp, we
can not use the corresponding component). Next, we make a
box whose 3 faces are the 3 faces of the selected component.
After that, we expand the distance between the 2 faces
for preshape/grasp so that the hand can not contact withthe box at preshape. In this expansion, the sizes of the 2
faces for preshape/grasp do not change while the other faces
are extended along the line orthogonal to the 2 faces forpreshape/grasp. We define the orientation of PB as follows:z axis is normal of the face for palm, and the direction frompalm location to the face is positive. x axis is normal of thefaces for preshape, and the direction from the face for thumbfinger to the face for the other fingers is negative. Then, theorientation of PB expresses which face for preshape is for
thumb finger and which face for preshape is for the otherfingers. The position of PB is set to the center of PB. If
the number of available PB (or components) is over one, we
select either of them, using a criterion such as the distance
between the PB and the hand.
For example, the first PB for task 1 is shown in Fig.3.
Note that if the length of the edges along z axis of the faces
for preshape is so long that the hand can not grasp the box
by contacting the fingertips with the center of the faces forpreshape, we will shorten the edges.
D. Preshape planning
Based on the PB, we derive desired fingertip positions
for preshape. We determine the desired fingertip positions
so that the position and orientation of the grasping plane arecoincident with the position and orientation of the PB. The
grasping plane is defined as follows. Firstly, we consider
a triangle or a segment which is the convex hull of thefingertips except for thumb fingertip. We set the positionof the grasping plane is the center between the triangle or
the segment and the thumb fingertip. x axis is set to thenormal vector of the triangle or the direction orthogonal to
the segment which goes through the thumb fingertip. y axis
is set to the direction from the ring fingertip to the index
fingertip. If the line parallel to x axis which goes throughthe thumb fingertip goes through the inside of the triangle or
the segment, force closure grasp can be obtained. If roughly
speaking, the grasping plane can expresses the object frame
constructed by the fingertips. Note that this discussion isfor the grasps with 4 fingers. We determine the number offingers for grasps according to the size of PB. We can definethe grasping plane by a similar way even if the number offingers for grasps is lower than 4.
Next, we derive hand base (palm) position and orientation,
based on the desired fingertip positions for preshape. Letph ∈ R3 , Rh ∈ R3×3 be the position and orientation of thehand base, and φh be the three components expressing Rh
such as roll, pitch and yaw angles. Let h pf i ∈ R3 be thefingertip position of the ith finger with respect to the handbase frame, and pf id be its desired position. Let q i be theangle vector of the ith finger. Then, the hand state can be
expressed by x = [pTh φTh q T1 q T2 q T3 q T4 ]T . The desired
fingertip position for the ith finger is expressed by
pf id = ph + Rh (φh )h pf i

(4)

Let xn be x whose q i is the current joint angle of ith
finger, and whose ph and φh are the position and orientationof PB. We linearize the relation (4) around the nominal handstate xn . Then, we can get the linear relation (A1 δx = b1 )with respect to δx which expresses the displacements fromthe nominal state to the state realizing the desired fingertip
positions. We also consider the finger constraints such as thelimitation of joint angles, which can be expressed by linear
inequalities (A2 δx ≤ b2 ). Then, we can get x (= xn + δx)
realizing the desired fingertip positions, solving the following
convex quadratic problems.
min δxT δxδxsubject toA1 δx = b1
A2 δx ≤ b2

(5)

Note that by solving this problem, we can get the desired
joint angles for the desired fingertip positions. Therefore, we
can use the desired joint angles for control. Note also that
hand base motion corresponds to arm motion. We derive arm
motion so that its tip, namely hand base can converge to its
desired position and orientation.
IV. M OTION PLANNING
Here, we propose a method to plan a motion for theplanned grasp by multi-fingered hand, which can avoid
any undesired collision, and can applied to regrasping anda motion with collision. For the purpose, we use vector
field based motion planning strategy [5]. For every obstacle(with which collision must be avoid), we make a potentialfunction which has high value if the robot is close to the
obstacle. We view the potential function as energy. Then,
its gradient expresses (reciprocal) force and its direction. It
is called vector field. If the robot has several components
such as several links, we sum the reciprocal forces for allcomponents. The summed force is the reciprocal force forthe robot. On the other hand, we also consider a potentialfunction for the desired state of the robot. The function
becomes small monotonously as the robot comes close toits desired state. We view its gradient as attractive force andits direction. Appling the both attractive and repulsive forces
to the robot, the robot can reach its desired state avoiding
collisions with the obstacle.

When applying this method to our target problem, we haveto resolve the following problems:
1) we can not use the summed reciprocal force. For
example, let us consider the case when grasping a box
by contacting thumb with a certain face and contacting
index and middle fingers with its antipodal face. If the
distance between the box face and the thumb is almost
same as the distance between the box face and theindex and the middle fingers, the summed reciprocal
force is not zero although it must be zero. Then, thereciprocal force arises the motion in which the thumbcollides with the box.
2) According to the reciprocal force, which part of thehand should be moved: finger joints or hand base(palm)?
3) How do we deal with regrasping and a motion withcollision?
First, we consider the bounding boxes for all links andpalm body of the hand, using CAD data of the hand.
SAHhand can be divided into 18 parts. Let bi (i = 1, · · · , 18)
be the corresponding bounding box. We make 3 groups
according to the faces of PB; first group B1 (= {b1 }) is for
palm, second group B2 (= {b2 , · · · , b6 }) is for thumb finger
corresponding to the face for thumb for preshape, last group
B3 (= {b7 , · · · , b18 }) is for the other fingers corresponding
to the other face for preshape. We calculate the distance
between bi and the box to be grasped or obstacle. If using L1
norm for distance, we can calculate it by linear programming
such as simplex method. Let di be the distance. We canget also the point on bi and the direction (from the pointon the box or obstacle to the point on bi ) which give di .
For every group, we find minimum distance (for example,
mini=2,··· ,6 di for B2 ), and the corresponding point on bi anddirection which give the minimum distance. We call the point(the nearest point to the box or obstacle) the reciprocal point.
Note that the number of the derived minimum distances
is not always one. For example, if considering group B2
and let dmin = mini=2,··· ,6 di , then, we regard di such that|di − dmin | ≤  where  is a small constant value as minimum distance. We calculate the vector expressing reciprocal
force for the group, using the minimum distances and thecorresponding directions. Here, let f rij (j = 1, · · · , mi ) be
the vector for the group Bi where mi denotes the number ofthe vectors (the number of the derived minimum distances).
Let prij be the corresponding reciprocal point for f rij . We
define the target component which should move according to
the reciprocal force. Joint angles and the frame fixed at handbase (hand base position and orientation) could be the target
component. Let Grij be the jacobian matrix expressing therelationship between prij and the target component. Then,
we calculate the reciprocal force with respect to the group
Bi as follows:mi

GTrij f rij /mi .

j=1

This reciprocal force is commensurately applied to the handwith respect to the group Bi . Therefore, we can resolve thefirst problem described at the initial part of this section. If thetarget component is the frame fixed at hand base, we sum thereciprocal forces for all the groups (B1 , B2 , B3 ), and apply
the summed force to the frame. If the target component is
joint angles, we consider only the reciprocal force for thegroup including the joint angles, and apply it to the joint
angles.
Next, we consider the second problem. Since the movable
region for finger is small, collision avoidance motion by
finger angles is limited to small area. Near the box to be
grasped, we do not always generate the suitable motion byhand base (palm). Let us consider the aforementioned casewhen grasping a box by thumb, index and middle fingers,
again. If unluckily the distance between the thumb and index
or middle finger is smaller than the size of the box, we needto open the fingers each other. This motion can be done
by finger angles, but can not be done by moving hand base.
Note that it is just an example, and barely happen in succededplanning. Here, we divide the motion into 3 parts: the motion
for bringing the hand to near the object, the motion forpreshaping, the motion for grasping. At the first motion, thereciprocal force is applied to only hand base. At the second
motion, the reciprocal force is applied to not only the handbase but also joint angles. If the hand is very close to its
desired state, the reciprocal force is applied to only joint
angles. At the third motion, we do not apply any reciprocal
forces.
At task 5 (Fig.1 (e)), the thumb finger must collide withthe box while the other fingers must avoid collision with thebox. We use the aforementioned 3 groups Bi (i = 1, 2, 3).
The group for thumb finger B2 can be regarded as the groupwhich will collide with the box. The other groups can be
regarded as the group which must avoid any collisions. At
the first motion, we use the same way as the way mentioned
the above. At the second motion, we take the following way:
Before collision (between the thumb and the box), we apply
the reciprocal forces to the hand base and the joint angles
contained at the group B3 . After collision, we apply thereciprocal force to the joint angles contained at the group B3 .
Then, the robot can move so that the thumb finger collides
with the box while the other fingers and bodies can avoid
any collisions.
We generate the motion for the hand holding the box suchthat the box face with the opening area can face the robot.
If the holding hand moves, the box also moves. Then, if
the other hand will grasp the moveable box, the motion forthe (other) hand must be consistent with the box motion.
We can get the command (motion) information for the handholding the box, since it is done by the robot itself. Using
this information, we derive the motion command for thehand which will grasp the (movable) box. We add the force
compensating for the attract and reciprocal forces for thehand holding box to the attract and reciprocal forces forthe other hand which will grasp the box. Then, we can getsuitable motion for the hand which will grasp the box; therobot moves to grasp a moveable box, avoiding undesired

(1)

(2)

(3)

(4)

(5)

(6)

(7)

(8)

(9)

(10)

(11)

(12)

(13)

(14)

Fig. 6.

Box opening taskcollisions with it.
Lastly, grasping is done by moving the fingers in the
direction of internal forces. Then, we can get the grasp
motion which do not affect (arise) the motion of the box.
The amount of finger motion for grasping without crushing isdetermined according to real experimental results. We checkthe force magnitude by torque sensors attached with every
joint to avid crashing. On the other hand, if crashing isneeded, we also determine the amount of finger motion byexperimental results.

V. N UMERICAL EXAMPLE
To conform the validity of our approach, we show anumerical example. We consider the system shown in Fig.2.
We don’t use a specified arm. The main program is written
by C++ language. For the simulation, we use openrave [19].
At the openrave environment, we get the information about
joint angles, hand base state (position and orientation) andobject state (position and orientation). As for the object state,
we assume that we can get the object state by camera. Before
finding the opening area, the information from the camera
does not have the information about the opening area. Afterfinding the opening area, the information from the camera
includes the information about where the opening area is.
Firstly, we send the information to the main program.
Based on the information, the main program calculates thecommands for the finger joints and the hand base state, andsend them to the openrave. Repeating this way, we simulated.
We use yrap program [20] for the communication. We checkcollisions at the openrave environment.
Fig.6 shows the task for opening box. The number shows
the order of the task. (1) and (2) show task 1, and (2)∼(5)
show task 2. (5)∼(11) show task 3. Here we show several
views to show the collision avoidance motion and how
to deal with the movable target box. (5)∼(7) show how
to deal with the movable target box. (7)∼(10) show thecollision avoidance motion. (5)∼(10) show the first motion
part described at section IV while (10) and (11) show thesecond motion part. As for the third motion part for grasping,refer to see (12) which shows the left hand holds the box.
(11) and (12) show task 4, (12)∼(14) show task 5. We do notshow deformed (opened) state of the box at (14), but (14)shows the appropriate hand configuration. The configuration
for grasping or preshaping is also shown in Fig.1.
VI. C ONCLUSION
This paper presented a method of grasp motion planningfor 2 arms with multi-fingered hands for opening paper box.
we proposed a task priority based scheme to plan graspingarea consistent with whole steps of the given task procedure.
Based on the concept of preshape, we took the planned
grasping area and derived desired fingertip positions andhand base position and orientation for preshape. Based on thevector field approach, we proposed a method to plan a motion
for the planned grasp by multi-fingered hands, which can
avoid any undesired collisions. This method can be applied
to regrasping and a motion with collision.
This is the first step of our goal. Our goal is to develop arobot system which can autonomously manipulate an object
in human environment. Then, we have many further problems. For example, the robot is asked to open the box. Firstof all, the robot must investigate/explore the environment by
sensors, and find and recognize where the target object isand where the obstacles are. It is still difficult problem inthe environment including many uncertainties. In addition,
if there are several (same) boxes, the robot has to know
the intention of the commander and select the box. Next,
according to the situation, the robot has to make a procedure

for the task. After that, this approach can be applied. To deal
with other everyday tasks, we need to extend this approach
to more generalized one. Furthermore, we need to combine
the methods to resolve these problems. These are our furture
works.
R EFERENCES[1] C. Kemp, A. Edsinger, and E. Torres-Jara, “Challenges for robot
manipulation in human environments,” IEEE Robotics and Automation
Magazine, vol. 14, no. 1, pp. 20–29, March 2007.
[2] CoTeSys (Cognition for Technical Systems), [Online]. Available: http://cotesys.in.tum.de/.
[3] D. Wren and R. Fisher, “Dextrous hand grasping strategies usingpreshapes and digit trajectories,” Proc. of the IEEE International
Conference on Systems, Man and Cybernetics, pp. 910–915, 1995.
[4] M. Prats, P. Sanz, and P. del Pobil, “Task-oriented grasping using
hand preshapes and task frames,” Proc. of the IEEE International
Conference on Robotics and Automation, pp. 1794–1799, 2007.
[5] H. Choset, K. M. Lynch, S. Hutchinson, G. Kantor, W. Burgard,
L. E. Kavraki, and S. Thrun, Principles of Robot Motion: Theory,
Algorithms, and Implementations. MIT Press, Cambridge, 2005.
[6] A. Hofhauser, C. Steger, and N. Navab, “Edge-based template matching and tracking for perspectively distorted planar objects,” International Symposium on Visual Computing, 2008.
[7] A. Bicchi, “Hands for dexterous manipulation and robust grasping: adifficult road towards simplicity,” IEEE Transactions on Robotics andAutomation, vol. 16, no. 6, pp. 652–662, December 2000.
[8] M. T. Mason, Mechanics of Robotic Manipulation.
MIT Press,
Cambridge, 2001.
[9] T. Watanabe and T. Yoshikawa, “Grasping optimization using arequired external force set,” IEEE Transactions on Automation Scienceand Engineering, vol. 4, no. 1, pp. 52–66, 2007.
[10] A. Miller, S. Knoop, H. Christensen, and P. Allen, “Automatic grasp
planning using shape primitives,” Proc. of the IEEE International
Conference on Robotics and Automation, pp. 1824–1829, 2003.
[11] A. Miller and P. Allen, “Graspit!: A versatile simulator for grasp
analysis,” ASME Int. Mechanical Engineering Congress & Exposition,
2000.
[12] K. Harada, K. Kaneko, and F. Kanehiro, “Fast grasp planning forhand/arm systems based on convex model,” Proc. of the IEEE International Conference on Robotics and Automation, pp. 1162–1168,
2008.
[13] A. Morales, T. Asfour, and P. Azad, “Integrated grasp planning andvisual object localization for a humanoid robot with five-fingered
hands,” Proc. of the IEEE/RSJ International Conference on Intelligent
Robots and Systems, pp. 5663–5668, 2006.
[14] D. Berenson, R. Diankov, K. Nishiwaki, S. Kagami, and J. Kuffner,“Grasp planning in complex scenes,” Proc. of the IEEE/RAS International Conference on Humanoid Robots, 2007.
[15] A. A. Cole, P. Hsu, and S. S. Sastry, “Dynamic control of sliding
by robot hands for regrasping,” IEEE Transactions on Robotics andAutomation, vol. 8, no. 1, pp. 42–52, February 1992.
[16] M. A. Farooqi and T. OMATA, “Online construction of themanipulation-model of an unknown object by a robot hand using theregrasping primitives,” Advanced Robotics, vol. 14, no. 2, pp. 135–
151, 2000.
[17] M. Yashima, Y. Shiina, and H. Yamaguchi, “Randomized manipulation
planning for a multi-fingered hand by switching contact modes,” Proc.
of the IEEE International Conference on Robotics and Automation, pp.
2689–2694, 2003.
[18] H. Liu, P. Meusel, N. Seitz, B. Willberg, G. Hirzinger, and M. Jin,
“The modular multisensory dlr-hit-hand,” Mechanism and Machine
Theory, vol. 42, no. 5, pp. 612–625, May 2007.
[19] openrave, [Online]. Available: http://openrave.programmingvision.
com/index.php?title=Main Page.
[20] yarp, [Online]. Available: http://eris.liralab.it/yarp/.