Journal of Robotics, Networking and Artificial Life, Vol. 2, No. 1 (June 2015), 22-25

Error Recovery of Pick-and-Place Tasks
in Consideration of Reusability of Planning
Akira Nakamura, Kazuyuki Nagata, Kensuke Harada and Natsuki Yamanobe

Intelligent Systems Research InstituteNational Institute of Advanced Industrial Science and Technology (AIST)
Central 2, 1-1-1 Umezono, Tsukuba, Ibaraki, 305-8568 Japan
E-mail: a-nakamura@aist.go.jp, k-nagata@aist.go.jp,
kensuke.harada@aist.go.jp, n-yamanobe@aist.go.jp
www.aist.go.jp

Abstract
Error recovery in robotic tasks is explored to enable robots to be used for complicated tasks. The authors’ error
recovery processes make use of the concepts of both task stratification and error classification. In this paper, thereusability of planning in error recovery is verified by using the typical pick-and-place tasks that are used in plantmaintenance and industrial production.
Keywords: manipulation skill, error recovery, task stratification, error classification

1. Introduction
In recent years, studies on robotic manipulation for
performing tasks in various fields have been conducted.
We have conducted many research studies on therobotic manipulation used to perform plant maintenancetasks and manufacture industrial products (Fig. 1).
These manipulation tasks tend to be complex,
necessitating that composition rules be devised for theentire work process.
By analyzing the assembly and disassembly
sequences performed by humans, we found that thosetasks tend to be composed of several significant motion
primitives. We call each motion primitive a “skill” andhave shown that most maintenance and production tasks
can be composed of a number of skills.1–3
Ideally, a robotic task has to be successfullycompleted as planned. However, in the actual tasks ofcomplicated plant maintenance and industrial
production, it is not rare for the execution of a task toterminate before completion. Therefore, error recovery
is an important research theme for robots that need toperform such real-world tasks.4, 5
We have explored error recovery in robotic tasks toenable robots to be used for complicated tasks.6–9 Ourerror recovery processes make use of the concepts ofboth task stratification and error classification, and thesetechniques are based on the concept of skills.
Considering the use of various recovery paths, thereusability of task planning may become an important
aspect of research on error recovery. In this paper, thereusability of planning in error recovery is verified byusing the typical pick-and-place tasks that are used inplant maintenance and industrial production.

Published by Atlantis Press
Copyright: the authors
22

2. Stratification of Tasks
project (MAX)…

Project

project (2)project (1)task (max)…

Tasktask (2)task (1)task (0)SkillprimitiveFig. 2 Manipulation hierarchy

Fig. 1 Maintenance robot
StartSensing(All-over task (max) )Modeling(All-over task (max) )Planning(All-over task (max) )※1

task (1)※1(0)
task (2)Classification of error※1

In an ideal environment, tasks are achieved without anyerrors occurring. In actual manipulation, however,errors often do occur from various causes. Our conceptof error classification and process flow with error
recovery in the task hierarchy are described in thissection. See Ref. 6 for more details.

task (0)( i0 )※1

task ((0)i0 )Figure 2 shows a hierarchy of manipulation tasks6. If weignore the servo layer, the skill layer, which consists ofelements such as the move-to-touch and rotate-to-levelskills, is located in the lowest layer called the task(0)layer. One tier above the task(0) layer is the task(1) layer.
Similarly, task(i+1) is composed of sequences of task(i)elements. The top layer, where the error recovery loop
is closed, is called task(max) and one tier above task(max) iscalled the project layer. The project layer might also behierarchized, but we will not discuss this here.
3. Error Recovery in Stratified Tasks

Class T (max+1)
Stopping the process andexecuting all necessary changes

Class 2 Changing theplanning parameters
Class 3 Changing themodeling parameters
Class T (1) Changing thesensing parameters※3
Re-planning(task( (1)i1 ) )Class 1(1)(max)

task ( imax , …………… , i1 )task ( imax )※3Modeling(task( (1)i1 ) )(0)

2.2. Stratification of tasks※3Class T (2) Executing thenecessary changes

Sensing(task( (1)i1 ) )rotate-to-level and rotate-to-insert, all of which play animportant part in those tasks. A specific task iscomposed of sequences of skill primitives such as thesethree skills. Moreover, many skills can be defined basedon modified versions of these three fundamental skills3.

Execution※2task (0)( i0 )

3.1. Classification of errors

Sensing
Modeling
Confirmation

The causes of manipulation failures can be attributable
to several kinds of errors. We group the error states into
several classes of errors: execution, planning, modelingand sensing, according to the possible causes6.
Merely remedying the causes of these errors does
not always solve the problem. It may be necessary toreturn to a previous step when the working environmentis greatly changed by the error.

error

correct

End
Accurate expression:※2(0)task (0)( i0 ) ＝ task ( imax , ……… , i0 ) ,※3(1)
task (1)( i1 ) ＝ task ( imax , …… , i1 )Fig. 3 Process flow with error recovery

This section explains our concept of skills and thestratification of tasks. See Refs. 1, 2 for more details.
2.1. Concept of skills
We analyzed human motions in such tasks as
disassembly and reassembly and found that themovements consisted of several significant motion
primitives. We call such motion primitives “skills 1, 2”.
We considered three fundamental skills: move-to-touch,

3.2. Error recovery based on classification
A generalized process flow of stratified tasks that takes
error recovery into account has been shown in Ref. 6.
Figure 3 is an illustration of the central portion of Fig.
10 in Ref. 6. This process is performed based on
recovery through a backward correction process. At theConfirmation step in each skill primitive task(0)(i0) , theresult is judged to be correct or a failure by an automatic

Published by Atlantis Press
Copyright: the authors
23

process or by a human operator. Error recovery isperformed using the following error classification.
Class 1: When the error is judged to be an execution
error, task(1)(i1) is executed again without correcting
the parameter .
Class 2: When the error is judged to be a planningerror, task(1)(i1) is executed again with a change in theplanning parameters .
Class 3: When the error is judged to be a modelingerror, task(1)(i1) is executed again with a change in themodeling parameters.
Class T(1): When the error is judged to be a sensingerror, task(1)(i1) is executed again with a change in thesensing parameters.
Class T(2): task(2)(i2) is executed again after theexecution of the necessary changes and returns to thestart of one tier above the task(1)(i1) layer.
::Class T(max): task(max)(imax) is executed again after theexecution of the necessary changes and returns to thestart of the (max - 1) tier above the task(1)(i1) layer.
Class T(max+1): When it is judged that too manychanges will be required, the process being executed
is interrupted and the process returns to the start of
the all-over task.
4.

working environment may be necessary for a Class 2
error, and additional geometry modeling of the object
and the tool may be necessary for a Class 3 error. And
additional geometry modeling of the working
environment and calibration of the vision system may
be necessary for a Class 4 error.
5. Planning with a View Toward Reusability
5.1. About reusability
Generally, a manipulation robot operates according to acomputer program that is based on an ordered plan ofoperation. The tasks performed by a manipulation robot
are varied and may involve such grasping tasks as
gripper closing and opening, and transfer-related tasks
such as lifting and approaching.
Similar tasks can performed using a similar
computer program. As such, in systems with error
recovery functions, similar tasks can be performed in
the recovery portions. Therefore, it is possible to use asimilar program for similar kinds of tasks for the totalsystem, not only for the main operation part but for theadditional recovery parts as well. A system with highreusability—one in which many of the same programsin the whole system could be reused including therecovery parts—would provide a very efficientapproach to system configuration.
5.2. How to improve reusability

Expanded Processes in Error Recovery

4.1. Forward correction process
To correct the robot’s motions at each step, a manual
operation module for robot control can be inserted in theterminal processing of each primitive motion. For
example, slight errors concerning the position andorientation of the object after transition and thecondition of the grasped object can be corrected8. This
process is recovery through a forward correction
process which differs from recovery through thebackward correction process described in 3.2.
4.2. Additional tasks
Additional tasks may be necessary in some cases toperform the corrections of Class 2, 3 and 4 errors (Fig.
4).7 For example, additional geometry modeling of theLet us explain a method for improving reusability byusing the tasks involved in repacking objects from alarge box into a small box in a physical distribution
scenario as shown in Fig. 5. See Ref. 9 for more details
on this task.
Failures may occur due to various causes when
performing the given tasks, and in various situations
such as grasping, lifting, carrying and packing.
The failure in which a plastic (PET) beverage bottle
is dropped during transfer is considered here. In mostcases, the dropped object ends up in some sort ofdisordered position, for instance falling over to ahorizontal posture from its original vertical orientation.
While an object with the shape of a PET bottle topples
in most cases, sometimes it remains standing upright.

Published by Atlantis Press
Copyright: the authors
24

Additional task

Class kChanging theparameters

6. Conclusions
HomeP10(=P1)

P1
Home
P5

Move-toapproach

P9
P6 Move-todestination

Departure

Pre-grasp
P4

Classification of error

Fig. 4 Additional task

Move-betweenreference frames

Lower

Lift

P7
Leave

Gripper-open
P2Approach
P3

P8Grasp

Small box
Large box

Fig. 5 Picking and placing task using a gripper

Error recovery involves grasping the object andmoving it to an objective destination. A movement
about the postures in the task means keeping the object
vertical when it is already vertical and standing upright,
and righting it back to the vertical from a toppled
horizontal position. In the original scenario, most pickand-place tasks will not require restoring to verticalposture from an existing vertical orientation. In otherwords, the task is performed by a pick-and-place
program that expresses a change in three degrees of
freedom from the position without changing theorientation. The process of the error recovery when thefailure occurred proceeds by planning the flow
including a change in the orientation of grasping,carrying with that change of posture, and packing. Thatis, the task is performed by a pick-and-place programthat expresses a change in six degrees of freedom of theposition and orientation.
It is undesirable to have many programs involved
even in a transfer task. This program represents one ofpick-and-place that expresses a change in six degrees offreedom of the position and orientation. The change
from the original scenario can be considered as changein three degrees of freedom of the position without achange in orientation.
Furthermore, it is desirable for the program to be
one that is usable in a wide area as well as in a local
domain. That is why calibration needs to be performed
in a wide area. Global calibration means that not only
the pick-and-place task of the original scenario but also
the processes of error recovery can be performed by thesame program. Reusability increases when as many
common programs as possible can be used for the total
manipulation tasks.

In this paper, we have discussed the reusability ofplanning in manipulation tasks including error recoveryprocesses. The techniques and validity of the reusabilityof such programs have been presented and verified
using pick-and-place tasks. Programming of the total
task including error recovery will be simplified by using
the same planning in the path of the original task and in
multiple recovery paths.
In the future, we will conduct further research on
optimum adjustment methods for the various parameters
used in the error recovery paths and for selecting
common programs. We will attempt to apply ourtechniques to actual maintenance robots.
References
1. T. Hasegawa, T. Suehiro and K. Takase, A model-based
manipulation system with skill-based execution, IEEE
Trans. Robot. Autom., 8(5) (1992) 535-544.
2. A. Nakamura, T. Ogasawara, T. Suehiro and H. Tsukune,
Skill-based backprojection for fine motion planning, inProc. IEEE/RSJ Int. Conf. on Intell. Robots Syst., (Osaka,Japan, 1996), 2, 526-533.
3. A. Nakamura, T. Ogasawara, K. Kitagaki and T. Suehiro,
Using robust and simplified geometric models in skillbased manipulation, in Proc. IEEE/RSJ Int. Conf. onIntell. Robots Syst., (Hawaii, USA, 2001), 1, 138-145.
4. B. R. Donald, Planning multi-step error detection andrecovery strategies, Int. J. Robot. Res., 9(1) (1990) 3-60.
5. P. Pastor, M. Kalakrishnan, S. Chitta, E. Theodorou andS. Schaal, Skill learning and task outcome prediction formanipulation, in Proc. IEEE Int. Conf. Robot. Autom.,(Shanghai, China, 2011), 3828-3834.
6. A. Nakamura and T. Kotoku, Systematization of error
recovery in skill-based manipulation, Artificial Life andRobotics, Springer, 14(2) (2009) 203-208.
7. A. Nakamura and Y. Kawai, Recovery technique fromclassified errors in skill-based manipulation, in Proc.
15th Int. Symp. on Artificial Life and Robotics (AROB
15th ’10), (Oita, Japan, 2010), 1010-1013.
8. A. Nakamura, K. Nagata, K. Harada, N. Yamanobe, T.
Tsuji, T. Foissotte and Y. Kawai, Error recovery using
task stratification and error classification for manipulation robots in various fields, in Proc IEEE/RSJ Int. Conf.
on Intell. Robots Syst., (Tokyo, Japan, 2013), 3535-3542.
9. A. Nakamura, K. Nagata, K. Harada and N. Yamanobe,
Error recovery using task stratification and error classification for manipulation robots in physical distribution,
Trans. Control and Mechanical Syst., 3 (2014) 122-131.

Published by Atlantis Press
Copyright: the authors
25