Visuospatial Skill Learning for Robots
S. Reza Ahmadzadeha,∗, Fulvio Mastrogiovannib , Petar Kormushevc
a School of Interactive Computing, Georgia Institute of Technology, Atlanta, GA, USA
of Informatics, Bioengineering, Robotics and Systems Engineering, University of Genova, via Opera Pia 13,
16145 Genova, Italy
c Dyson School of Design Engineering, Imperial College London, London SW7 2AZ, United Kingdom

b Department

arXiv:1706.00989v1 [cs.RO] 3 Jun 2017

Abstract
A novel skill learning approach is proposed that allows a robot to acquire human-like visuospatial skills
for object manipulation tasks. Visuospatial skills are attained by observing spatial relationships among
objects through demonstrations. The proposed Visuospatial Skill Learning (VSL) is a goal-based approach
that focuses on achieving a desired goal configuration of objects relative to one another while maintaining
the sequence of operations. VSL is capable of learning and generalizing multi-operation skills from a single
demonstration, while requiring minimum prior knowledge about the objects and the environment. In contrast
to many existing approaches, VSL offers simplicity, efficiency and user-friendly human-robot interaction. We
also show that VSL can be easily extended towards 3D object manipulation tasks, simply by employing point
cloud processing techniques. In addition, a robot learning framework, VSL-SP, is proposed by integrating
VSL, Imitation Learning, and a conventional planning method. In VSL-SP, the sequence of performed
actions are learned using VSL, while the sensorimotor skills are learned using a conventional trajectorybased learning approach. such integration easily extends robot capabilities to novel situations, even by users
without programming ability. In VSL-SP the internal planner of VSL is integrated with an existing actionlevel symbolic planner. Using the underlying constraints of the task and extracted symbolic predicates,
identified by VSL, symbolic representation of the task is updated. Therefore the planner maintains a
generalized representation of each skill as a reusable action, which can be used in planning and performed
independently during the learning phase. The proposed approach is validated through several real-world
experiments.
Keywords: robot learning, visuospatial skill, imitation learning, learning from demonstration, visual
perception,

1. Introduction
During the past two decades, several robot skill
learning approaches based on human demonstrations have been proposed (Ijspeert et al., 2013;
Argall et al., 2009). Many of them address motor skill learning in which new sensorimotor skills
are transferred to the robot using policy derivation
techniques. Motor skill learning approaches can be
categorized into two main groups: trajectory-based

and goal-based. To emulate the demonstrated skill,
the former group put the focus on recording and
regenerating trajectories (Ijspeert et al., 2013) or
intermittent forces (Kronander and Billard, 2012).
For instance, Bentivegna et al. (2004) proposed a
method in which a humanoid robot learns to play
air hockey by learning primitives such as velocity
and positional trajectories. By combining Reinforcement Learning with Imitation Learning, Kormushev et al. (2010) taught a robot to flip pancakes,
bootstrapping the learning process with a few observed demonstrations.

∗ Corresponding

author
Email addresses: reza.ahmadzadeh@gatech.edu (S.
Reza Ahmadzadeh), fulvio.mastrogiovanni@unige.it
(Fulvio Mastrogiovanni), p.kormushev@imperial.ac.uk
(Petar Kormushev)

In many cases, however, it is not the trajectory
that is of central importance, but the goal of the
task (e.g. solving a jigsaw puzzle). Learning every

single trajectory in such tasks actually, increases
the complexity of the learning process unnecessarily (Niekum et al., 2012). To address this drawback, the latter group of approaches which focus on
the goal of the task has been proposed (Verma and
Rao, 2005; Dantam et al., 2012; Chao et al., 2011).
There is a large body of literature on grammars
from the Computational Linguistic and Computer
Science communities, with a number of applications
related to Robotics (Niekum et al., 2012; Dantam
et al., 2012). Furthermore, a number of symbolic
learning approaches exist, which focus on goal configuration rather than action execution (Chao et al.,
2011). However, in order to ground the symbols,
they comprise many steps inherently, namely segmentation, clustering, object recognition, structure
recognition, symbol generation, syntactic task modeling, motion grammar, and rule generation. Such
approaches require a significant amount of a priori
knowledge to be manually engineered in the system
(Niekum et al., 2012; Dantam et al., 2012). In addition, most above-mentioned approaches assume the
availability of the information on the internal state
of a tutor such as joint angles, which cannot be
accessed easily by humans to imitate the observed
behavior.
An alternative to motor skill learning is visual
learning. These approaches are based on observing
demonstrations and using human-like visual skills
to replicate a task (Kuniyoshi et al., 1994; Lopes
and Santos-Victor, 2005).
We propose a novel visual skill learning approach
for interactive robot learning tasks. Unlike motor
skill learning approaches, our framework utilizes visual perception as the main source of information
for learning new skills from a demonstration. The
proposed approach, which we refer to as Visuospatial Skill Learning (VSL), uses visuospatial skills
to replicate the demonstrated task. A visuospatial skill is the capability to visually perceive the
spatial relationship between objects. VSL requires
minimum a priori knowledge to learn a sequence of
operations from a single demonstration. In contrast
to many previous approaches, VSL exhibits simplicity, efficiency, and user-friendly human-robot interaction. Rather than relying on complicated models
of human actions, labeled human data, or object
recognition, our approach allows a robot to learn
a variety of complex tasks effortlessly, simply by
observing and reproducing the visual relationship
among objects. We demonstrate the feasibility of
the proposed approach in several experiments in

which the robot learns to organize objects of different shape and color on a tabletop workspace
to accomplish a goal configuration. In the conducted experiments the robot acquires and reproduces main capabilities such as absolute and relative object placement, classification, turn-taking,
user intervention to modify the reproduction, and
multiple operations performed on the same object.
In Section 4, we show that, by employing point
cloud processing techniques instead of 2D image
processing methods, VSL can be extended for object manipulation tasks in 3D space. The new variant of VSL is called VSL-3D.
Integrating VSL and a conventional motor learning approach such as Imitation Learning enables
a robot to acquire new sensorimotor skills directly
from demonstrations. In Section 5, we show that by
this integration, the need for a classical equationbased trajectory generation module can be avoided.
In addition, in order to present the learned skills in
a symbolic form, a standard action-level planner
is combined with VSL. Learning preconditions and
effects of the actions in VSL and making symbolic
plans based on the learned knowledge deals with
the long-standing important problem of Artificial
Intelligence, namely Symbol Grounding (Harnad,
1990). This framework is denoted as VSL-SP and
is described in Section 5.
In the rest of this paper, we use the term VSL
for the original version of our proposed method, explained and used for 2D experiments in Section 3.
We also use the term VSL-3D for the 3D extension of VSL, explained in Section 4. The proposed
framework by integrating VSL, Imitation Learning,
and symbolic planning explained in Section 5 is referred to as VSL-SP.
The paper is organized as follows. Related work
is reviewed in Section 2. The terminology and
methodology of the proposed approach together
with a few implementation steps are explained in
details in Section 3. Results for simulated and realworld experiments are reported in Section 3.4 and
Section 3.5, respectively. VSL-3D, the extension of
the proposed approach for dealing with 3D applications, is presented in Section 4. Results reported in
Section 4.2 show that VSL-3D is readily attainable,
although it requires more sophisticated image processing techniques. VSL-SP, the integrated framework, comprising a motor skill learning approach,
VSL, and a symbolic planner, is described in Section 5. The implementation steps for the proposed
framework are discussed in Section 5.5. The fea2

sibility and capability of the proposed framework
are experimentally validated and the results are reported in Section 5.6. Conclusions of the paper are
drawn in Section 6.

Yeasin and Chaudhuri (2000) proposed a similar
approach that focuses on extracting and classifying subtasks for grasping tasks. Their approach
captures data by tracking the tutor’s hand during demonstrations, generates trajectories, and extracts subtasks using neural networks.
A visual learning by imitation approach presented by Lopes and Santos-Victor (2005), utilizes
neural networks and viewpoint transformation to
map visual perception to visuo-motor processes.
They adopted a Bayesian formulation for gesture
imitation. The method proposed by Pardowitz
et al. (2006) extracts knowledge about a pick-andplace task from a sequence of actions. They extract
the proper primitive actions and the constraints of
the task from demonstrations.
Pastor et al. (2009) presented an approach to
encoding motor skills into a non-linear differential
equation that can reproduce those skills afterward.
Object-Action Complex (Krüger et al., 2009) was
used to provide primitive actions with semantics.
However, the resulting symbolic actions were never
used in a high-level planner.
Ekvall and Kragic (2008) proposed a symbolic
learning approach in which a logical model for an
STRIPS-like planner is learned from multiple human demonstrations. For each object, a geometric
model and a set of features are given to the algorithm. The method can only deal with rectangular
objects and the precision of the positioning is decided using the K-means clustering method. The
spatial relations model presented by Golland et al.
(2010) has several limitations which prevent it from
being implemented on a real robot. For instance,
the approach assumes a perfect and noiseless visual information and a perfect object segmentation. Furthermore, a small grammar is used, which
means that the system just works for a few carefully
constructed expressions.
Chao et al. (2011) proposed a symbolic goalbased approach for grounding discrete concepts
from continuous perceptual data using unsupervised learning.
Task goals are learned using
Bayesian inference. The approach was implemented
on tasks including a single pick-and-place operation
in which the initial objects configuration is randomized.
Mason and Lopes (2011) used verbal commands
to instruct a robot to perform a room tidying task.
Primitive actions are predefined and the approach
uses a database including all possible discrete configurations of objects. The features are learned us-

2. Related Work
Visual skill learning or learning by watching is
one of the most powerful mechanisms of learning
in humans. It has been shown that even infants
can imitate both facial and manual gestures (Meltzoff and Moore, 1983). Learning by watching has
been investigated in Cognitive Science as a source
of higher order intelligence and fast acquisition
of knowledge (Rizzolatti et al., 1996; Park et al.,
2008). In the next paragraph, related and most
recent approaches on the topic of robot learning
using visual perception are discussed. Mentioned
works include those that (a) can be categorized as
goal-based, (b) utilize visual perception as the main
source of information, and (c) especially those focusing on learning object manipulation tasks.
One of the most influential works on plan extraction has been discussed by Ikeuchi and Suehiro
(1994). By obtaining and segmenting a continuous
sequence of images their method extracts assembly
plans. To extract transitions between sets of object
relations, their approach depends on a model of the
environment and predefined coordinate systems.
The early work of Kuniyoshi et al. (1994) focused
on acquiring reusable high-level task knowledge by
watching a demonstration. Their approach employs
a model of the environment, a predefined action
database, multiple vision sensors and basic visual
features. Their approach cannot detect rotations
and can only deal with rectangular objects which
are not in contact with each other.
Asada et al. (2000) proposed an approach based
on epipolar constraint for reconstructing the robot’s
view, on which adaptive visual servoing is applied
to imitate an observed motion. Instead of using coordinate transformation, two sets of stereo cameras
are used, one for the robot and the other for the
demonstrator.
Ehrenmann et al. (2001) used multiple sensors
for learning pick-and-place tasks in a kitchen environment. A magnetic field based tracking system,
an active trinocular camera head, and a data glove
were used in their experiments. Detected hand configurations were matched with a predefined symbol database detected using pre-trained neural networks.
3

ing a beta regression classifier and the planner finds
a solution using simulated annealing. The conducted experiments are of particular value for us
because they can be easily learned and reproduced
using our approach.
Kroemer and Peters (2011) proposed a two-layer
framework for learning manipulation tasks. One
layer optimizes the parameters of motor primitives,
whereas the other relies on pre- and post-conditions
to model actions. In contrast to VSL, pre- and postconditions are manually engineered in the system.
Aksoy et al. (2011) showed how symbolic representations can be linked to the trajectory level using
spatiotemporal relations between objects and hands
in the workspace. In a similar way, Aein et al.
(2013) showed how such grounded symbolic representations can be employed together with low-level
trajectory information to create action libraries for
imitating observed actions.
There is a large body of literature on grammars
from the Computational Linguistic and Computer
Science communities, with a number of applications
related to Robotics (Niekum et al., 2012; Dantam
et al., 2012). Niekum et al. (2012) proposed a goalbased method for learning from trajectory-based
demonstrations. Segmentation and recognition are
achieved using a Beta-Process Autoregressive Hidden Markov Model (HMM), while Dynamic Movement Primitives are used to reproduce the skill.
The approach needs to assign a coordinate frame
to each detected object and it cannot deal with rotations. Increasing the number of operations in the
demonstrations makes the segmentation process increasingly more prone to errors. Sometimes during
the segmentation process, extra skills can be extracted mistakenly.
Dantam et al. (2011) proposed Motion Grammar
to plan the operation of a robot through a contextfree grammar. The method was used to learn
human-robot chess games in which a point cloud
of the chessboard was used to detect the pieces.
Dantam et al. (2012) presented an approach for
the visual analysis of demonstrations and automatic
policy extraction for an assembly task. They used
several techniques such as segmentation, clustering,
symbol generation and abstraction. Guadarrama
et al. (2013) proposed a natural language interface
for grounding nouns and spatial relations. The system uses a classifier trained using a huge database
of collected images for each object. The presented
applications and the spatial representations are limited to 2D information.

By introducing a stack-based domain specific language for describing object repositioning tasks, Feniello et al. (2014) built a framework based on our
initial VSL approach (Ahmadzadeh et al., 2013b).
By performing demonstrations on a tablet interface,
the time required for teaching is reduced and the reproduction phase can be validated before execution
of the task in the real-world. Various types of realworld experiments have been conducted including
sorting, kitting, and packaging tasks.
There is a long history of research in the area of
interpreting spatial relations (Burgard et al., 1999;
Kelleher et al., 2006; Moratz and Tenbrink, 2006;
Skubic et al., 2004). In most of these works, a
model of spatial relations is built by hard-coding
the meaning of the spatial relations. Instead, some
recent works concluded that a learned model of
spatial relations can outperform hard-coded models (Golland et al., 2010; Guadarrama et al., 2013).
Visuospatial Skill Learning (VSL), is a goal-based
approach that utilizes visual perception as the main
source of information. It focuses on achieving the
desired goal configuration of objects relative to one
another while maintaining the sequence of operations. We show that VSL is capable of learning
and generalizing multi-operation skills from a single demonstration while requiring minimum prior
knowledge about the objects and the environment.
We also show that VSL can be extended towards
3D object manipulation tasks, simply by employing point cloud processing techniques. In contrast
to many existing approaches, VSL offers simplicity, efficiency and user-friendly human-robot interaction. In addition, we propose a learning framework (VSL-SP) by integrating, imitation learning,
VSL, and a conventional planning method. By integrating VSL with a conventional trajectory-based
learning approach, we show that primitive actions
can be learned directly through demonstrations instead of being programmed manually in the system.
The proposed integrated approach easily extends
and adapts robot capabilities to novel situations,
even by users without programming ability. To utilize the advantages of standard symbolic planners,
in VSL-SP the internal planner is integrated with
an existing action-level symbolic planner. The planner represents a symbolic description of the skills.
It uses the underlying constraints of the task and
extracted symbolic predicates (i.e. action preconditions and effects), identified by VSL, it updates the
symbolic representation while the skills are being
learned. Therefore the planner maintains a gener4

alized representation of each skill as a reusable action, which can be used in planning and performed
independently during the learning phase. We validate our approach through several real-world experiments with a Barret WAM manipulator and an
iCub humanoid robot.
3. Introduction to Visuospatial Skill Learning
Visuospatial Skill Learning (VSL) is a goal-based
robot Learning from Demonstration approach. A
tutor demonstrates a sequence of operations on a
set of objects. Each operation consists of a set of
actions (e.g. pick and place). In this Section, we
consider pick-and-place object manipulation tasks
in which achieving the goal of the task and retaining the sequence of operations are particularly important. In Section 5, we also consider other actions such as push and pull. We assume the virtual
experimental setup illustrated in Figure 1a, which
consists of a robot manipulator equipped with a
gripper, a tabletop workspace, a set of objects, and
a vision sensor. The sensor can be mounted above
the workspace to observe the tutor performing the
task. Using VSL, the robot learns new skills from
demonstrations by extracting spatial relationships
among objects. Afterward, starting from a random
initial objects configuration, the robot can perform
a new sequence of operations resulting in reaching
the same goal as the one demonstrated by the tutor. A high-level flow diagram, shown in Figure 1b,
illustrates the two main phases of VSL, namely
demonstration and reproduction. In the demonstration phase, for each action, a set of observations
is recorded used for the match finding process during the reproduction phase. During the match finding process, observations including the similar set
of features are matched. In this Section, first, the
basic terms for describing VSL are defined. Then,
the problem statement is described and finally, the
VSL approach is explained.

(a) Virtual experimental setup for a VSL task consisting
of a robot manipulator, a vision sensor, a set of objects,
and a workspace.

Demonstration
Action
Recording

Observation

Reproduction
Loading
Action

Match finding

(b) A high-level flow diagram of VSL illustrating the
demonstration and reproduction phases.
Figure 1: Virtual setup and flow diagram for VSL.

• Frame: a bounding box defining a rectangle
in 2D space or a cuboid in 3D space. The size
of the frame can be fixed or variable. The maximum size of the frame is equal to the size of
the world.
• Observation: the captured context of the
world from a predefined viewpoint using a specific frame. An observation can be a 2D image
or a 3D point cloud.

3.1. Terminology
In order to understand VSL, it is necessary to
introduce a few terms used throughout the paper.
These are in order:

• Pre-action observation: an observation
captured just before the action is executed.
The robot searches for preconditions in the preaction observations before selecting and executing an action.

• World : the workspace of the robot observable
by the vision sensor. The world includes objects being used during the learning task, and
reconfigured by the human tutor and the robot.

• Post-action observation: an observation
captured just after the action is executed. The
robot perceives the effects of the executed ac5

tions in the post-action observations.

a vertical borderline can be used to divide the
workspace into two areas illustrating right and left
zones. A landmark can be either static or dynamic.
A static landmark is fixed with respect to the world
during both demonstration and reproduction. A
dynamic landmark, however, can be replaced by the
tutor before the reproduction phase starts. Both
types of landmarks are shown in this paper. In
case that any landmarks (e.g. label, borderline) are
used in the demonstration phase, the robot should
be able to detect them in the world (line 2). However, we assume that the robot cannot manipulate
a landmark.
During the demonstration phase, VSL captures
one pre-action observation (OP re ) and one postaction observation (OP ost ) for each operation executed by the tutor using the specified frame (FD )
(lines 5,and 6). We assume that during each operation only one object can be affected by an action. The pre-action and post-action observations
are used to detect the object on which the action
is executed, the pose of the object before and after
the action execution (P P re , P P ost ) (line 7). VSL is
capable of detecting objects during the demonstration process without having any a priori knowledge
in advance. Practically, this can be achieved using
simple image processing techniques such as imagesubtraction and thresholding applied on a couple of
raw observations. For each detected object, a symbolic representation (B) is created (line 7). Such
symbolic representation of an object can be used
in case the reproduction phase of VSL is integrated
with a high-level symbolic planner as shown in Section 5. In addition, VSL extracts a feature vector
(φ) for each detected object (line 7). In order to
extract φ, any feature extracting method can be
used. In this Section, we use Scale Invariant Feature
Transform (SIFT) algorithm (Lowe, 2004). SIFT
is one of the most popular feature-based methods
which is able to detect and describe local features
that are invariant to scaling and rotation.
Pose vectors and the detected landmarks (L) are
used to identify preconditions and effects of the executed action through spatial reasoning (line 8). For
instance, if P P re is above a horizontal borderline
and P P ost is below the line, the precondition of the
action is that the object is above the line and the effect of the execution of the action is that the object
is below the line. By observing the predicates of an
executed action, the action itself can be identified
from the set of actions A (line 9). The sequence of
identified actions is then stored in a policy vector

The set of actions contains different primitive
skills such as pick, place, push, or pull. VSL assumes that actions are known to the robot and the
robot can execute each action when required. In
Section 5, VSL-SP extends VSL by teaching the
primitive actions to the robot through demonstrations (Ahmadzadeh et al., 2015).
3.2. Problem Statement
Formally, we define visuospatial skill learning as
a tuple
V = {W, O, F, A, C, Π, φ} ,
(1)
where W ∈ Rm×n is a matrix which represents the
context of the world including the workspace and
all objects (WD and WR refer to the world during
the demonstration and reproduction phases, respectively). O is a set of observation dictionaries O =

OP re , OP ost ; OP re and OP ost are observation
dictionaries comprising a sequence of pre-action
and post-action observations, respectively such
that: OP re = hOP re (1), OP re (2), . . . , OP re (η)i,
and OP ost = hOP ost (1), OP ost (2), . . . , OP ost (η)i,
where η is the number of operations performed by
the tutor during the demonstration phase. As a
consequence, OP re (i) represents the pre-action observation captured during the ith operation. F ∈
Rm×n is an observation frame used to capture observations. A is a set of primitive actions defined
in the learning task (e.g. pick). C is a set of constraint dictionaries C = C P re , C P ost ; C P re and
C P ost are constraint dictionaries comprising a sequence of pre-action, and post-action constraints,
respectively. Π is a policy or an ordered action sequence extracted from demonstrations. φ is a vector containing extracted features from observations
(e.g. SIFT features).
3.3. Methodology
Pseudo-code of VSL is sketched in Algorithm 1.
At the beginning (line 1), objects are randomly
placed in the world (WD ) and the size of the frame
(FD ) is equal to the size of the world (WD ). We
assume that the robot has knowledge of a set of
primitive actions A and it is capable of executing
them. For instance, the robot is capable of moving towards a desired given pose and execute a pick
action. In Section 5, we explain how a primitive
action library can be built.
In some tasks, to specify different spatial concepts, the tutor can use a landmark. For instance,
6

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16

Input : {W, F, A}
Output: {O, P, Π, C, B, φ}
Initialization: place objects in the world randomly
L=detectLandmarks(W)
i=1
// Part I : Demonstration
for each Action do
OiP re = getPreActionObs(WD , FD )
OiP ost = getPostActionObs(WD , FD )
[Bi , PiP re , PiP ost , φi ] = getObject(OiP re , OiP ost )
[CiP re , CiP ost ] = getConstraint(Bi , PiP re , PiP ost , L)
Πi = getAction(A, CiP re , CiP ost )
i=i+1
end
// Part II : Reproduction
for j = 1 to i do
Pj∗P re = findBestMatch (WR , OjP re , φj , CjP re , L)
Pj∗P ost = findBestMatch (WR , OjP ost , φj , CjP ost , L)
executeAction(Pj∗P re , Pj∗P ost , Πj )
end

Algorithm 1: Pseudo-code for VSL.

lastly P is a pure projective matrix. An affine
matrix, TA , that is the remainder of αT by extracting P can be calculated as TA = αTP−1 . T
is extracted by taking the 3rd column of TA and
A ∈ R2×2 , is the remainder of TA . A can be further
decomposed using SVD such that, A = UDVT ,
where D is a diagonal matrix, and U and V are
orthogonal matrices. Finally, we can calculate




 T

D 0
V
0
UVT 0
Sv =
, Rθ =
, Rφ =
.
0 1
0
1
0
1

Π.
During the reproduction phase, VSL is able to execute the learned sequence of actions independent
of any external action planner (Ahmadzadeh et al.,
2013a,b). In such case, VSL observes the new world
(WR ) in which the objects are replaced randomly.
Comparing the recorded pre- and post-action observations, with the new world, VSL detects the best
matches for each objects, considering the extracted
features, new location of the landmarks, and the
extracted constraints from the learning phase (lines
13 and 14). Although the findBestMatch function
can use any metric (e.g. window search method)
to find the best matching observation, to be consistent, in all of our experiments in this Section, we use
SIFT. Afterward, we apply Random Sample Consensus method (RANSAC) in order to estimate the
transformation matrix Tsift from the set of matches.
A normalization constant is used to have a unique
normalized representation and avoiding unnatural
interpolation results. The normalized matrix αTsift
can be decomposed into simple transformation elements,
αTsift = TRθ R−φ Sv Rφ P,

In a pick-and-place task, since Rθ is calculated
ick
lace
for both pick and place operations (RP
, RP
),
θ
θ
the pick and place rotation angles of the objects are
extracted as follows:
 P ick

R
(2,2)
θP ick = arctan RPθ ick (2,1)
(2)
 Pθlace

R
(2,2)
θP lace = arctan RθP lace (2,1) .
(3)
θ

VSL relies on vision, which might be obstructed
by other objects, by the tutor’s body, or during the
reproduction by the robot’s arm. Therefore, for the
physical implementation of the VSL approach special care needs to be taken to avoid such obstructions. Figure 2 shows the result of match finding
using SIFT applied to an observation and a new
world.
After finding the best match, the Algorithm extracts the pose of the object before and after action

where R±φ are rotation matrices to align the axis
for horizontal and vertical scaling of Sv ; Rθ is another rotation matrix to orientate the shape into
its final orientation; T is a translation matrix; and
7

place point, Pj∗P lace , while the hand is rotating to
meet the place rotation angle, θjP lace . Then, the
robot places the object in the place point and finally moves back along the blue curve to the rest
point. In order to form each trajectory, initial conditions (i.e. initial positions and velocities) and a
specific duration must be defined. Thereby, a geometric path is defined which can be expressed in
the parametric form of the following equations:

Observation

Match Finding

Subtraction

px =

World

py =
pz =

Figure 2: The result of the image subtracting and thresholding for a place action (right), match finding result using SIFT
in the 4th operation during reproduction of the Domino task
(left).

a3 s3 + a2 s2 + a1 s + a0
3

2

b3 s + b2 s + b1 s + b0
−1

h[1 − |(tanh

κ

(h0 (s − 0.5))) |],

(4)
(5)
(6)

where, s is a function of time t, (s = s(t)), px =
px (s), py = py (s), and pz = pz (s) are the 3D elements of the geometric spatial path; The ai and bi
coefficients in (4) and (5) are calculated using the
initial and final conditions. κ in (6) is the curvature,
h0 and h are the initial height and the height of the
curve in the middle point of the path respectively.
h and h0 can be either provided by the tutor or detected through depth information provided by an
RGB-D sensor. In addition, the time is smoothly
distributed with a 3rd order polynomial between the
tstart and tf inal which both are instructed by the
user. Moreover, to generate a rotation angle trajectory for the robot’s hand, a trapezoidal profile is
used together with the θP ick and θP lace calculated
in (2) and (3). As shown in Figure 3a, the trajectory generation module can also deal with objects
placed in different heights (different z-axis levels).
It is worth noting that to reproduce the task with
more general capabilities, a generic symbolic planner can be utilized instead of the reproduction part
of the Algorithm. Such extension is described in
Section 5.
Another vital element for implementing a pickand-place operation is the grasping strategy. There
are many elaborate ways to do grasp synthesis for
known or unknown objects (Bohg et al., 2014; Su
et al., 2012). Since the problem of grasping is
not the main focus of our research, we implement
a simple but efficient grasping method using the
torque sensor of the robots used in the experiments.
The grasping position is calculated using the center
of the corresponding pre-action observation. The
grasping module firstly opens all the fingers and
closes them after the hand is located above the desired object. The fingers stop closing when the measured torque is more than a pre-defined threshold
value. In addition, by estimating a bounding box

execution, Pj∗P re , Pj∗P ost (lines 13 and 14). Finally,
an action is selected from the policy Π and together
with pre and post poses is sent to the executeAction function (line 15). This function selects the Aj
primitive action.
To perform a pick-and-place operation, the robot
must execute a set of primitive actions consisting of reaching, grasping, relocating, and releasing. Either one of the reaching and relocating actions are trajectory-based skills which can either
be manually programmed to the system or a tutor
can teach them to the robot for instance through
learning by demonstration technique (Ahmadzadeh
et al., 2015) as shown in Section 5. In the original
VSL (Ahmadzadeh et al., 2013b), a simple trajectory generation strategy was employed that we explain here. The extracted pick and place points
(Pj∗P ick , Pj∗P lace ) together with the corresponding
rotation angles (θjP ick , θjP lace ), are used to generate
a trajectory for the corresponding operation. For
each pick-and-place operation, the desired Cartesian trajectory of the end-effector is a cyclic movement between three key points: a rest point, a pick
point, and a place point. Figure 3a illustrates a
complete trajectory cycle generated for a pick-andplace operation. Also, four different profiles of rotation angles are depicted in Figure 3b. The robot
starts from the rest point while the rotation angle is equal to zero and moves smoothly along the
red curve towards the pick point, Pj∗P ick . During
this movement, the robot’s hand rotates to satisfy
the pick rotation angle, θjP ick , according to the rotation angle profile. Then the robot picks up an
object, relocates it along the green curve to the
8

‘house body’. The goal of this experiment is to show
the VSL’s capability of disambiguation of multiple
alternative matches. If the algorithm uses a fixed
search frame (FR ) that is smaller than the size of
the ‘bodies’ (i.e. blue objects in the world ), then,
as shown in the first and second sub-figures 4b, the
two captured observations can become equivalent
(i.e. O1 = O2 ) and the 3rd operation might be
performed incorrectly (see the incorrect reproduction in Figure 4c). The reason is that, due to the
size of the frame, the system perceives a section
of the world not bigger than the size of a ‘house
body’. The system is not aware that an object is
already assembled on the top and it will select the
first matching pre-place position to place the last
object there. To resolve this problem we use adaptive size frames during the match finding process in
which the size of the frame starts from the smallest
feasible size and grows up to the size of the world.
The function findBestMatch in Algorithm 1, is responsible for creating and changing the size of the
frame adaptively in each step. This technique helps
the robot to resolve the ambiguity issue by adding
more context inside the observation, which in effect
narrows down the possible matches and leaves only
a single matching observation. Figure 4c shows the
sequence of reproduction including correct and incorrect operations.

(a) A full cycle of spatial trajectory generated for a pickand-place operation.

Angle of Rotation [deg]

100
80
60
40
20
0
−20
−40
−60
−80
−100

0

1

2

3

4

5

6

7

8

time [sec]
(b) The generated angle of rotation, θ, for the
robot’s hand.
Figure 3: The generated trajectory cycle including position
and orientation profiles for a spatial pick-and-place task.

3.5. Real-world Experiments
In this Section, five real-world experiments are
described. As it can be seen in Figure 5, the
setup for all the experiments consists of a torquecontrolled 7 DOF Barrett WAM robotic arm
equipped with a 3-finger Barrett Hand, a tabletop
working area, a set of objects, and a CCD camera
mounted above the workspace (not necessarily perpendicular to it). The resolution of the captured
images is 1280 × 960 pixels. In all the conducted
experiments, the robot learns simple object manipulation tasks including pick-and-place actions. In
order to perform a pick-and-place operation, the
extracted pick and place poses are used to make a
cyclic trajectory as explained in Section 3.3. In the
demonstration phase, the size of the frame for the
pre-pick observation is set equal to the size of the
biggest object in the world, and the size of the frame
for the pre-place observation two or three times bigger than the size of the biggest objects in the world.
In the reproduction phase, the size of the frame is
set equal to the size of the world. Note that, in
all the illustrations, the straight and curved arrows

for the target observation, the values are used to
decide which axis is more convenient for grasping.
3.4. Simulation
In this Section, a simulated experiment is designed to gain an understanding of how VSL operates and to show the main idea of VSL without
dealing with practical limitations and implementation difficulties. For this simulation, as shown in
Figure 4, a set of 2D objects is made which the tutor can manipulate and assemble them on an empty
workspace using keyboard or mouse. Each operation consists of a pick and a place action, which are
executed by holding and releasing a mouse button.
In this task, the world includes two sets, each
containing three visually identical objects (i.e.
three blue ‘house bodies’ and three brown ‘roofs’).
As it can be seen in Figure 4a, the tutor selects the
‘roof’ objects arbitrarily and places them on top of
the ‘bodies’. However, in the 3rd operation, the tutor intentionally puts the ‘roof’ at the bottom of the
9

1

2

3

(a) Demonstration

(b) Three steps of adaptive frame-size
Figure 5: The experimental setup for a VSL task.

1

2

to detect multi-object movements. Such extension
is out of the scope of the presented work.

-- incorrect
- correct
3
3

Alphabet Ordering
In the first task, the world includes four cubic
objects labeled with A, B, C, and D letters. The
world also includes a static right angle baseline as
a landmark (L). The goal is to reconfigure and sort
the set of objects with respect to the baseline according to a previous demonstration. As reported
in Table 1, this task emphasizes VSL’s capability
of the relative positioning of an object with respect
to other surrounding objects (a visuospatial skill).
This is achieved through the use of visual observations capturing both the object of interest and
its surrounding objects (i.e. its context). In addition, the baseline is provided to show the capability of absolute positioning of VSL. It shows the
fact that we can teach the robot to attain absolute
positioning of objects without defining any explicit
a priori knowledge. Figure 6a shows the sequence
of operations in the demonstration phase. Recording pre-pick and pre-place observations, the robot
learns the sequence of operations. Figure 6b shows
the sequence of operations produced by VSL starting from a novel world (i.e. a new initial configuration) achieved by randomizing objects locations
in the world. The accompanying video shows the
execution of this task (Video, 2013a).

(c) Reproduction
Figure 4: Roof placement simulated experiment to illustrate
VSL’s capability of disambiguation of multiple alternative
matches.

are used just to show the sequence of operations,
not the actual trajectories for performing the movements. Table 1 summarizes different capabilities of
VSL emphasized in each task.
In order to test the repeatability of VSL and to
identify the possible factors of failure, the captured
observations from the real-world experiments were
used while excluding the robot from the loop. All
other parts of the loop were kept intact and each
experiment was repeated three times. The result
shows that less than 5% of pick-and-place operations failed. The main failure factor is the match
finding error which can be resolved by adjusting
the parameters of SIFT-RANSAC or using alternative match finding algorithms. The noise in the images and the occlusion of the objects can be listed
as two other potential factors of failure. Despite
the fact that VSL is scale-invariant, color-invariant,
and view-invariant, it has some limitations. For instance, if the tutor accidentally moves one object
while operating another, the algorithm may fail to
find a pick/place position. One possible solution is
to combine classification techniques together with
the image subtraction and thresholding techniques

Animal Puzzle
In this task, the world includes two sets of objects which complete a ‘frog’ and a ‘giraffe’ puzzle. There are also two labels (i.e. landmarks) in
the world, a ‘pond’ and a ‘tree’. The goal is to
10

Task

Table 1: Capabilities of VSL illustrated in each real-world experiment.

Capability

Relative positioning
Absolute positioning
Classification
Turn-taking
User intervention

Animal
Puzzle
X
-

Alphabet
Ordering
X
X
-

Tower of
Hanoi
X
X
X
X

Animals vs.
Machines
X
X
X

Domino
X
X
X

1

4

2

3

2

3

4
1

(a) The sequence of the operations in the demonstration phase
by the tutor

(a) The sequence of the operations in the demonstration phase
by the tutor

3
1

1

2
3

4

2

4

(b) The sequence of the operations in the reproduction phase
by the robot

(b) The sequence of the operations in the reproduction phase
by the robot

Figure 6: Alphabet ordering. The initial configuration of the
objects in the world is different in 6a and 6b. Black arrows
show the operations.

Figure 7: Animal puzzle. The initial and the final objects’
configurations in the world are different in (a) and (b). Black
arrows show the operations.

assemble the set of objects for each animal with
respect to the labels according to the demonstration. Figure 7a shows the sequence of operations in
the demonstration phase. To show the capability
of generalization, the ‘tree’ and the ‘pond’ labels
are randomly replaced by the tutor before the reproduction phase. Figure 7b shows the sequence
of operations reproduced by VSL after learning the
spatial relationships among objects. This experiment shows the VSL’s capability of relative positioning reported in Table 1. In the previous task,
the final objects’ configuration in the reproduction
and the demonstration phases are always the same.
In this experiment, however, by removing the fixed
baseline from the world, the final result can be a
totally new configuration of objects. The accompanying video shows the execution of this task (Video,
2013a).

the entire stack to another rod. This experiment
demonstrates almost all capabilities of VSL. Two of
these capabilities are not accompanied by the previous experiments. Firstly, our approach enables the
user to intervene to modify the reproduction. Such
capability can be used to move the disks to another
base (e.g. to move the stack of disks to the third
base, instead of the second). This can be achieved
only if the user performs the very first operation in
the reproduction phase and moves the smallest disk
on the third base instead of the second. Secondly,
VSL enables the user to perform multiple operations on the same object during the learning task.
The sequence of reproduction is shown in Figure 8.
Animals vs. Machines: A Classification Task
This is an interactive task that demonstrates the
VSL capability of classification of objects. In this
task, we show that interestingly the same VSL algorithm can be used to learn a classification task.
The robot is provided with four objects, two ‘animals’ and two ‘machines’. Two labeled bins are
used in this experiment for classifying the objects.
Similar to previous tasks, the objects, labels, and

Tower of Hanoi
In this experiment, the famous Tower of Hanoi
puzzle, which consists of a number of disks of different sizes and three bases or rods which actually are
landmarks. The objective of the puzzle is to move
11

1

5

2

6

3

7

4

8

Figure 8: The sequence of the reproduction for the Tower of
Hanoi experiment to illustrate the main capabilities of VSL.

Figure 9: The sequence of operations in the demonstration
phase. Each column represents one pick-and-place operation. In each operation, the tutor picks one object and classifies it either as an ‘animal’ or a ‘machine’. The selected
object in each operation is shown in the middle row.

bins are not known to the robot initially. First,
all the objects are randomly placed in the world.
The tutor randomly picks objects one by one and
places them in the corresponding bins. Then, in
the reproduction phase, the tutor places one of the
objects each time, in a different sequence with respect to the demonstration. Differently from previous tasks, the robot does not follow the operations sequentially but searches in the pre-pick observation dictionary for the best matching pre-pick
observation. The selected pre-pick observation is
used for reproduction. The tutor can modify the
sequence of operations in the reproduction phase
by presenting the objects to the robot in a different order with respect to the demonstration. The
sequence of operations in the demonstration phase
is illustrated in Figure 9. Each row represents one
pick-and-place operation. During each operation,
the tutor picks an object and moves it to the proper
bin. The set of pre-pick and pre-place observations
can be seen in left and right columns, respectively.
Figure 10 shows two operations during the reproduction phase. The accompanying video shows the
execution of this task (Video, 2013b).

phase, the tutor first demonstrates all the operations. In order to learn the spatial relationships,
the system uses the modified algorithm from the
classification task. In the reproduction phase, the
tutor starts the game by placing the first object
(or another) in a random place. The robot then
takes the turn, finds and places the next matching domino piece. The tutor can also modify the
sequence of operations in the reproduction phase
by presenting the objects to the robot in a different order with respect to the demonstration. The
sequence of reproduction is shown in Figure 11.
4. VSL-3D: An Extension to 3D Tasks
In this Section, an extension encompassing VSL
is discussed. Although the main structure of the algorithm remains unaltered, VSL-3D requires more
sophisticated image processing methods including
3D feature estimation, calculation of surface properties, dealing with noise, 3D match finding, and
pose estimation.

Domino: A Turn-taking Task
The goal of this experiment is to show that VSL
can also deal with the tasks involving the cognitive
behavior of turn-taking. In this task, the world includes a set of objects all of which are rectangular
tiles. Each two pieces of the puzzle fit together to
form an object (Figure 11). In the demonstration

4.1. Point Cloud Processing
For a VSL-3D task, point cloud processing methods should be used in both demonstration and reproduction phases. In the demonstration phase,
for each operation, VSL-3D captures one pre-action
12

rigid transformation, different sampling density,
and noise. In this section, Fast Point Feature Histogram (FPFH) proposed by Rusu et al. (2009) is
used which is a computationally efficient version of
Point Feature Histogram. FPFH encodes a point’s
k-neighborhood geometrical properties by generalizing the mean curvature around the point using
a multi-dimensional histogram of values. FPFH is
chosen because it is 6D pose-invariant and copes
with varying sampling densities or noise levels, it
depends on the quality of the surface normal estimate at each point. The match finding and
pose estimation process for 3D experiments are accomplished by utilizing three different algorithms
consisting of Iterative Closest Point (ICP), Prerejective RANSAC (Buch et al., 2013), and SAmple
Consensus Initial Alignment (SAC-IA) (Rusu et al.,
2009). Figure 13 demonstrates some results for the
match finding process.
The main differences for implementing VSL in
2D versus 3D are reported in Table 2. The table
includes the techniques used in this paper. However, the implementation of VSL is not limited to
these techniques and depending on the application,
various image processing and point cloud processing techniques can be employed.

Figure 10: Two operations during the reproduction phase.
Red crosses on objects and on bins show the detected positions for pick and place actions, respectively.

and one post-action clouds. The captured raw point
clouds are filtered using a pass-through filter to cut
the values outside the workspace. This action reduces the number of points by removing unnecessary information (i.e. distant points) from the raw
point clouds. The limits of filtering for each axis are
derived from the workspace size. From each pair
of filtered point clouds, pre-action and post-action
observations are generated. In this state, to increase the speed of further processing, point clouds
are downsampled through voxelizing. The points in
a voxel are approximated with their centroid. To
avoid decreasing the data resolution, the leaf size of
the voxel grid must be selected appropriately. The
points generated in voxelization phase are stored
in a Kd -tree structure. Then, K Nearest-Neighbor
search (KNN) is applied to extract the downsampled point cloud by finding correspondence between
groups of points. The search precision (i.e. error
bounds) and the distance threshold should be set
properly for the search algorithm. The result of applying the described process on a pair of captured
observations is depicted in Figure 12.
In the reproduction phase, each captured world
observation (WR ) after being voxelized is compared
with the corresponding recorded observations in
the dictionaries and a metric is applied to find
the best match. Surface normals are estimated for
each cloud using a Kd -tree and search for neighboring points in a pre-defined radius. The estimated surface normals are then used to find local features. There are several methods to represent point features (Steder et al., 2010). An
acceptable feature descriptor should be able to
show the same characteristics in the presence of

4.2. Results
In this section, the feasibility and capability of
VSL-3D are experimentally validated. The setup is
similar to the one described in Section 3.5, where
the CCD camera is replaced with an RGB-D sensor
namely, Asus Xtion Pro Live.
Table Cleaning
In the first experiment with VSL-3D, the world
includes an empty box and a set of objects which
are randomly placed in the workspace. The goal is
to clean up the table by picking objects and placing two of them inside the empty box and the final
one in front of the box according to the demonstration. The tutor demonstrates the task and the
observations are captured by the sensor. Before the
reproduction phase starts, the objects are reshuffled
randomly in the workspace. Figure 14 shows the sequence of operations reproduced by the robot. Although in some operations the objects are occluded
by the front wall of the box, the robot can accomplish the task successfully.
13

Figure 11: The sequence of reproduction performed by the robot and the tutor are shown for the turn-taking task of domino.
Table 2: The main differences between VSL and VSL-3D approaches.

VSL

VSL-3D

Observations
Applicable for
Transformation

2D Image
3DoF Reconfiguration (X, Y, θ)
2D to 2D (3 × 3)

Capturing
observations

Background Subtraction
Thresholding

Match finding

2D Features
2D Feature Estimation
2D Metrics (SIFT)
RANSAC
No overlap among objects
Objects with the same height
Objects placed at the same level

3D Point Cloud
6DoF Reconfiguration
3D to 3D (4 × 4)
Voxelizing
Kd -tree
KNN search
3D Features
3D Feature Estimation
Voxelizing
Normal/Curvature Estimation
3D Metrics (ICP/RANSAC/SAC-IA)
Partially visible objects
Objects with different heights
Objects placed in different levels

Main
assumptions

finished, the robot can reproduce the stacking task
in the workspace even if the objects and the boxes
are randomly placed. The corresponding sequence
of operations is shown in Figure 15b.
Results
In both experiments, the pre- and post-action
observations are extracted from a single demonstration. These observations are point cloud data
which are actually 2.5D and are acquired from a
single point of view. Since VSL does not use a priori knowledge about objects, if reconfigured in the
world, they might not be detected and recognized
successfully during the reproduction phase. If an
object is observed in the demonstration phase, there
is no guarantee that it can be detected in the reproduction phase while observed from a new point of
view. This is due to object transformations before
the reproduction phase. Thus, in order to implement a robust version of VSL-3D, more robust features for both textured and textureless objects and
modifications in the point cloud processing part of
the algorithm are needed. However, the main steps
of the algorithm remain unaltered. Since image processing and point cloud processing are not the focus

Figure 12: Result of a subtraction process between a pre-pick
(top-left) and a pre-place (bottom-left) point clouds. The
original clouds are voxelized at a resolution of 2 mm and the
distance threshold for the KNN-search is set to 1 cm.

Stacking Objects
In the next pick-and-place task, objects are randomly placed on the table (the world ). The scene
includes two dynamic landmarks (i.e. two boxes)
which are repositioned by the tutor before starting
the reproduction phase. In the first phase, the tutor
starts stacking the objects on top of the two boxes.
The sequence of operations is captured and together
with the initial and final object configurations are
shown in Figure 15a. After the learning phase is
14

(a) RGB-D

(b) ICP

(c)

Pre-rejective RANSAC

(d) SAC-IA

Figure 13: (a) illustrates the captured RGB-D cloud of a scene. The result of the match finding process using different methods
are shown in (b), (c), and (d).

Figure 14: The sequence of operations reproduced by the robot for the table cleaning task.

of this paper, we direct the readers towards appropriate references, e.g. (Ekvall and Kragic, 2008;
Papazov et al., 2012; Guadarrama et al., 2013).

of the actions using VSL. We discuss this solution
in this section.
Furthermore, in order to use the advantage of
standard symbolic planners, the original planner
of VSL can be replaced with an already available
action-level symbolic planner, for instance, SGPlan
6 by Chen et al. (2004). The integration of VSL
and a symbolic planner brings significant advantages over using VSL alone. Learning preconditions
and effects of the actions in VSL and making symbolic plans based on the learned knowledge deals
with the long-standing important problem of Artificial Intelligence, namely Symbol Grounding (Harnad, 1990).
There are a few approaches in which a highlevel symbolic representation is formed using lowlevel action descriptions. The method proposed
by Jetchev et al. (2013) finds relational, probabilistic STRIPS operators by searching for a symbol grounding that maximizes a metric balancing
transition predictability with model size. Konidaris
et al. (2014) avoid this search by directly deriving
the necessary symbol grounding from the actions.
In this Section, a new robot learning framework
is proposed by integrating a sensorimotor learning framework (Imitation Learning), VSL, and a
symbolic-level action representation and planning

5. Learning Symbolic Representations of
Actions using VSL
As described in Section 3 and Section 4, VSL
and VSL-3D allow a robot to identify the spatial
relationships among objects and learn a sequence
of actions for achieving the goal of a task. We
have shown that VSL is capable of learning and
generalizing different skills such as object reconfiguration, classification, and turn-taking interactions
from a single demonstration. One of the shortcomings of VSL is that the primitive actions such as
pick, place, and reach, have to be manually programmed in the robot. For instance, in Section 3.3,
a simple trajectory generation module was devised
that is able to produce trajectories for pick and
place primitive actions. This issue can be addressed
by combining VSL with a trajectory-based learning
approach such as Imitation Learning (Ijspeert et al.,
2001). The obtained framework can learn the primitive actions of the task using conventional Learning from Demonstration strategies while it learns
the sequence of actions, preconditions, and effects
15

(a) Demonstration

(b) Reproduction
Figure 15: Demonstration and reproduction phases in a stacking task. The initial and final object configurations are shown in
top rows. The bottom rows, from left to right, show the sequence of operations.

• Symbolic Planning (SP), which uses a symbolic representation of actions, to plan or replan the execution of the task. A symbolic
planner allows the system to generalize not
only to different initial states, but also to different final states.

layer. The capabilities and performance of the proposed framework, which is called VSL-SP, are validated using real-world experiments with an iCub
robot.
5.1. Overview
The proposed learning approach, VSL-SP, consists of three layers:

A high-level flow diagram of VSL-SP including
the three layers together with their input and output can be seen in Figure 17. IL is employed to
teach sensorimotor skills to the robot (e.g. pull and
push). Learned skills are stored in a primitive action library, which is accessible by the planner. VSL
captures an object’s context for each demonstrated
action. This context is the basis of the visuospatial
representation and encodes implicitly the relative
position of the object with respect to other objects
simultaneously. In order to employ the advantages
of symbolic-level action planners, a PDDL 3.0 com-

• Imitation Learning (IL), which is suitable
for teaching trajectory-based skills (i.e. actions) to the robot, as it has been posited
by Argall et al. (2009). IL relaxes the assumption about having to encode actions manually.
• Visuospatial Skill Learning (VSL), which
allows us to capture spatial relationships
among an object and its surrounding objects
and to extract a sequence of actions to reach
the goal of the task.
16

converts desired position, velocity, and acceleration,
y, ẏ, and ÿ, into motor commands. DMP employs a
damped spring model that can be modulated with
nonlinear terms such that it achieves the desired
attractor behavior, as follows:
τ ż = αz (βz (g − y) − z) + f + Cf ,
τ ẏ = z.

(7)

In (7), position and velocity are represented by
y and z, respectively, τ is a time constant and αz
and βz are positive constants. With βz = αz /4,
y monotonically converges toward the goal g. f is
the forcing term and Cf is an application dependent coupling term. Trajectories are recorded independently of time. Instead, the canonical system is
defined as:
τ ẋ = −αx x + Cc ,
(8)

Figure 16: The iCub robot interacting with objects.

where αx is a constant and Cc is an application dependent coupling term. x is a phase variable, where
x = 1 indicates the start time and x close to zero
means that the goal g has been reached. Starting
from some arbitrarily chosen initial state x0 such as
x0 = 1 the state x converges monotonically to zero.
The forcing term f in (7) is chosen as follows:
PN
ψi (x)ωi
x(g − y0 ),
(9)
f (x) = Pi=1
N
i=1 ψi (x)

pliant planner is integrated with IL and VSL to provide a general purpose representation of a planning
domain. The identified preconditions and effects
modify their formal PDDL definition (i.e. the planning domain). Once formal action definitions are
obtained, it is possible to exploit the planning domain to reason upon scenarios that are not strictly
related to what has been learned, e.g. including
different object configurations, a varied number of
objects or different objects at all. One of the main
advantages of VSL-SP is that it extracts and utilizes multi-modal information from demonstrations
including both the sensorimotor information and visual perception, which is used to build symbolic
representation structures for actions.

where ψi , i = 1, . . . , N, are fixed exponential basis
functions, ψi (x) = exp(−hi (x − ci )2 ), where σi and
ci are the width and centers of the basis functions,
respectively, and wi are weights. y0 is the initial
state. The parameter g is the target coinciding with
the end of the movement g = ydemo (t = P ), y0 =
ydemo (t = 0). The parameter τ must be adjusted
to the duration of the demonstration. The learning
process of the parameters wi is accomplished with
a locally weighted regression method, because it is
very fast and each kernel learns independently of
others (Ijspeert et al., 2013).
In the conducted experiments two primitive actions, namely pull and push are used, each of which
is demonstrated to the robot through Kinesthetic
teaching and learned using DMP. The robot can reproduce the action from a new initial pose towards
the target. To achieve this goal, the pull action
is decomposed into reach after and move backward
sub-actions and also the push action is decomposed
into reach before and move forward sub-actions.
The recorded sets of demonstrations for both reach
before and reach after sub-actions are depicted as

5.2. Learning Sensorimotor Skills using Imitation
Learning
IL enables robots to learn and reproduce
trajectory-based skills from a set of demonstrations
through kinesthetic teaching (Ijspeert et al., 2013).
In VSL-SP, IL is utilized to teach primitive actions (i.e. pull and push) to the robot. In particular, Dynamic Movement Primitives (DMP), which
are designed for modeling attractor behaviors of
autonomous nonlinear dynamical systems, are utilized (Ijspeert et al., 2013).
In order to create a pattern for each action, multiple desired trajectories in terms of position, velocity and acceleration have to be demonstrated and
recorded by a human operator, in the form of a vector [ydemo (t), ẏdemo (t), ÿdemo (t)], where t ∈ [1, ..., P ]
and P is the number of datapoints. A controller
17

Figure 17: A flow diagram illustrating the proposed integrated approach, VSL-SP, comprising three main layers. The robot
learns to reproduce primitive actions through Imitation Learning. It also learns the sequence of actions and identifies the
constraints of the task using VSL. The symbolic planner is employed to solve new symbolic tasks and execute the plans.

Figure 18: The tutor is teaching the primitive actions including pull and push to the robot using kinesthetic teaching.

black curves in Figure 19. Each red trajectory in
Figure 19 illustrates a reproduction. In both Figures, the goal (i.e. the object), is shown in green.
Finally, the learned primitive actions are stored in
the primitive action library. Later, the planner connects the symbolic actions to the library of actions.
The move forward and move backward sub-actions
are straight line trajectories that can be learned or
implemented directly into the trajectory generation
module.

0.2

z[m]

0.15
0.2

z[m]

0.1

0.15

0.05

0.1
0.05

0
0

0
0.15

0.05

0.15
0.1

0.1

y[m]
0

5.3. Learning Action Sequences by VSL
At this stage, a library of primitive actions has
been built. The robot learns the sequence of actions required to achieve the goal of the demonstrated task through VSL. It also identifies spatial
relationships among objects by observing demonstrations. As shown in Figure 17, both IL and VSL
observe the same demonstrations (not necessarily at
the same time) but utilize different types of data.
IL uses sensorimotor information (i.e. trajectories),
whereas VSL uses visual data.

-0.06
-0.04
-0.02

x[m]

(a) reach-before sub-action.

0.1
0.05

0.05

y[m]
0

0

x[m]

(b) reach-after sub-action.

Figure 19: The recorded set of demonstrations for two subactions are shown in black. The reproduced trajectories from
an arbitrary initial position towards the target (the green
square) are shown in red.

be represented as a symbolic, discrete, scenarioindependent form. To this aim, a symbolic representation for both the pull and push actions based
on the PDDL 3.0 formalism is defined. However,
it is necessary to learn preconditions and effects for
each action in the primitive action library. The sensorimotor information in the skill learning process
is exploited to derive symbolic predicates, instead
of defining them manually.

5.4. Generalization of Learned Skills as Symbolic
Action Models
In order to use the learned primitive actions for
general-purpose task planning, the actions need to
18

pushed and pulled. Specifically, push(?b) is an action that makes the predicate truth value switching
from ¬(far ?b) to (far ?b), whereas the opposite
holds for pull(?b). Visual information about the
location of objects in the World is mapped to the
proper truth value of the (far ?b) predicate. The
execution of a push demonstration on a green cube
(as processed by VSL) allows the system to understand that before the action ¬(far Bgreen ) holds,
after the action (far Bgreen ) holds, and the two
predicates contribute to the :precondition and
:effect lists of a push action defined in PDDL.
The second class includes constraints (in the form
of PDDL predicates) related to the trajectory of the
plan that is executed in the demonstration. This
can be done analyzing the sequence of actions as
represented in the VSL model (i.e. the implicit
chain defined by predicates in OP ost and OP re observation dictionaries). In PDDL 3.0, it is possible to define additional constraints, which implicitly
limit the search space when the planner attempts to
find a solution to the planning problem. In particular, given a specific planning domain (in the form of
push and pull actions, as obtained by reasoning on
the VSL model), the problem can be constrained
to force the planner to follow a specific trajectory
in the solution space. One possible constraint is to
impose, for instance, the sequence of satisfiability
of predicate truth values. As an example, let us assume the VSL model represents the fact that Bgreen
must be always pushed after Bblue . According to
our formal definition, this implies that the predicate (far Bgreen ) must hold sometime after the
predicate (far Bblue ) holds. To encode this constraint in PDDL formalism, it is sufficient to use the
constraint (somewhere-after (far Bgreen ) (far
Bblue )). If the constraint is used in the planning problem, the planner attempts to find a plan
where this sequence is preserved, thereby executing push(Bgreen ) strictly after push(Bgreen ). Finally, it is noteworthy that such a constraint is removed, the planner is free to choose any suitable
sequence of actions to be executed (independently
of the demonstration), provided that the goal state
is reached.

As shown in Figure 20, the robot first perceives
the initial workspace in which one object is placed
in far. Then the robot is asked to execute the pull
action, and after the action is finished, the robot
perceives the workspace again. The pre-action and
post-action observations are shown in Figure 20.
The robot uses VSL to extract the preconditions
and effects of the pull action considering the landmark in the workspace (i.e. the black line in this
case). The same steps are repeated for the push
action. The domain file in the PDDL formalism is
updated automatically by applying this procedure.
The preconditions and effects for the pull and push
actions are reported in Table 3.

Figure 20: By extracting the preconditions and effects of the
primitive actions while executing them, the robot generalizes
the learned sensorimotor skills as symbolic actions.

Table 3: The preconditions and effects extracted by VSL and
written to the planner’s domain file.

Action
precondition
effect

push
¬ (far ?b)
(far ?b)

pull
(far ?b)
¬ (far ?b)

5.5. Implementation
In this step, it is possible to generalize each action as well as the demonstrated action sequences,
using a PDDL 3.0 compliant formalism. It is noteworthy that in the present work, symbolic knowledge has not been bootstrapped from scratch. On
the contrary, starting from the knowledge of the
performed action types during the demonstration,
symbolic-level knowledge is updated with two kinds
of constraints. The former class includes constraints (in the form of PDDL predicates) related
to preconditions and effects. This is done by mapping the elements of observation dictionaries OP re
and OP ost to relevant predicates. In the considered scenario, one predicate is enough to characterize push and pull actions, namely (far ?b), where
far is the predicate name and ?b is a variable that
can be grounded with a specific object that can be

5.6. Results
In order to evaluate the capabilities and performance of VSL-SP, a set of experiments are performed with an iCub humanoid robot. As shown
in Figure 16, the robot is standing in front of a
tabletop including some polyhedral objects. All
19

alent sensorimotor skills. Initially, there are three
objects on the table. The tutor provides the robot
with a sequence of actions depicted in Figure 22a.
The robot perceives the demonstration, records the
observations and extracts the sequence of actions
using VSL. In the reproduction phase, objects are
reshuffled in the workspace. For the reproduction of
the task, instead of SP (Symbolic Planner), the internal planner of VSL is exploited. The result shows
that starting from an arbitrary initial configuration
of the objects in the workspace, VSL is capable of
extracting the ordered sequence of actions and reproducing the same task as expected. As shown in
Figure 22b, the robot achieves the goal of the task
using VSL. The accompanying video shows the execution of this task (Video, 2015).

the objects have the same size but different colors and textures. First, the tutor teaches a set
of primitive actions, including pull and push, to
the robot using Imitation Learning. For each action, a set of demonstrations is captured through
Kinesthetic teaching, while the robot is perceiving the workspace (Figure 18). Using Imitation
Learning, the robot learns to reproduce each action
from different initial positions of the hand towards
the desired object. The learned primitive actions
are stored in a library. In addition, to introduce
the concept of far and near, a black landmark,
which represents a borderline, is placed in the middle of the robot’s workspace. The robot can detect the landmark using conventional edge detection and thresholding techniques. Also, the size of
the frames (FD , FR ) are defined equal to the size of
the world. Figure 21 shows some instances of image
processing results from the conducted demonstrations. In this Section, blob detection and filtering
by color techniques are utilized by considering that
the iCub robot is capable of detecting the height of
the table at the beginning of each experiment by
touching it. To detect the object which is moved
and the place that the object has been moved to,
background subtraction is used. Moreover, a simple
(boolean) spatial reasoning has been applied that
utilizes the detected pose of the object and the detected borderline and then decides that the object
is far or near. However, to extract more sophisticated constraints, qualitative spatial reasoning languages such as Region Connection Calculus by Randell et al. (1992) can be employed.

a

b

c

d

e

f

Keeping All Objects Near
In the previous experiment, it has been shown
that the internal planner of VSL is capable of reproducing the task. However, VSL-SP is more capable in generalizing the learned skill. In this experiment, the robot learns a simple game including
an implicit rule: keep all objects near. For each object, two properties are defined: color and position.
The symbolic planner provides VSL-SP with the capability of generalizing over the number and even
shape of the objects. In order to utilize this capability, the tutor performs a set of demonstrations
to eliminate the effect of color implying that any
object independent of its color should not be far
from the robot. Three sets of demonstrations are
performed by the tutor, which are shown in Figure 23. Each demonstration starts with a different initial configuration of objects. The performed
demonstrations imply the following rules:
r11 : hBgreen → neari IF hBorange is fari
r12 : hBorange → neari IF hBgreen is neari
r21 : hBgreen → neari IF hBorange is neari

(10)

r31 : hBorange → neari IF hBgreen is fari

Figure 21: (a) and (b) are rectified pre-action and postaction observations; (c) is a rectified pre-action observation
for the next operation; (d) is obtained by background subtraction between (a) and (b); (e) is obtained by background
subtraction between (b) and (c); (f) is the result of detecting
the border line for spatial reasoning.

where rij indicates the j th rule extracted from the
ith demonstration. The first demonstration includes two rules r11 , r12 because two actions are executed. It can be seen that the right parts of r11
and r21 and the right parts of r12 and r31 eliminate
each other. Also, the color attribute on the left
side of the rules eliminates the effect of the color
on action execution. During the demonstration,

A Simple VSL Task
This experiment evaluates the obtained connection between the symbolic actions and their equiv20

(a) Demonstration.

(b) Reproduction.

Figure 22: A simple VSL task. In both sub-figures, the bottom rows show robot’s point of view.

for each object, the robot extracts the spatial relationships between the objects and the borderline,
and decides if the object is far or near (i.e. not
far). It then applies the extracted precondition of
r11 : hB? → neari to that object. Afterward, the
robot is capable of performing the learned skill on
a various number of objects and even unknown objects with different colors. The accompanying video
shows the execution of this task (Video, 2015).

the green one, and the green cube before the yellow one. The robot has to learn that the most important cube has to be operated first. It has been
shown that learning a sequence of actions in which
the order is important, is one of the main capabilities of VSL. Therefore, if the task was demonstrated
once and the reproduction part was done by VSL
(and not by VSL-SP) the robot would learn the task
using one single demonstration. In addition, VSL
provides the robot with the capability of generalizing the task over the initial objects’ configuration.

Furthermore, using VSL-SP, it is possible to encode such an ordering also in a planning domain, at
the cost of adding extra constraints (in the form of
predicates) to the definition of the planning problem. In this case the planner should include two
extra predicates, namely (somewhere-after (far
Bgreen ) (far Borange )) and (somewhere-after
(far Byellow ) (far Bgreen )). The inclusion of
these predicates must be managed at an extra-logic
level, e.g. by extracting the sequence of actions
using VSL and inserting the corresponding constraints in the planner problem definition. In this
case, there is no need to have more demonstrations,
but expressing the order of sequence as preconditions may be done automatically. The accompanying video shows the execution of this task (Video,
2015).

Figure 23: Three different sets of demonstrations devised by
the tutor for implying the rule of the game in the second
experiment.

Pull All Objects in a Given Order
In the previous experiment, the robot learns that
all objects must be kept close, without giving importance to the order of actions. In this experiment, to give priority to the sequence of actions to
be performed, the color attribute for each object
is included as well. For instance, consider the ordered set horange, green, yellowi in which the orange cube and the yellow cube are the most and
least important cubes, respectively. This means
that we want the orange cube to be moved before
21

demonstrations. In contrast to many existing approaches, VSL leverages simplicity, efficiency, and
user-friendly human-robot interaction. The feasibility and efficiency of VSL have been validated
through simulated and real-world experiments.
References
References
Figure 24: In the third experiment, using VSL-SP, the robot
needs six sets of demonstrations for each of which the initial
(a) and final (b) objects’ configurations are shown. Using
VSL only the robot requires a single demonstration for which
the sequence of operations is shown (a-d).

Aein M.J., Aksoy E.E., Tamosiunaite M., Papon J., Ude
A., Worgotter F. Toward a library of manipulation actions based on semantic object-action relations. In: Intelligent Robots and Systems (IROS), IEEE/RSJ International Conference on. IEEE; 2013. p. 4555–62.
Ahmadzadeh S.R., Kormushev P., Caldwell D.G. Interactive robot learning of visuospatial skills. In: Advanced Robotics (ICAR), 16th International Conference
on. IEEE; 2013a. p. 1–8.
Ahmadzadeh S.R., Kormushev P., Caldwell D.G. Visuospatial skill learning for object reconfiguration tasks. In:
Intelligent Robots and Systems (IROS), IEEE/RSJ International Conference on. IEEE; 2013b. p. 685–91.
Ahmadzadeh S.R., Paikan A., Mastrogiovanni F., Natale
L., Kormushev P., Caldwell D.G. Learning symbolic representations of actions from human demonstrations. In:
Robotics and Automation (ICRA), IEEE International
Conference on. Seattle, Washington, USA: IEEE; 2015.
p. 3801–8.
Aksoy E.E., Abramov A., Dörr J., Ning K., Dellen B.,
Wörgötter F. Learning the semantics of object–action
relations by observation. The International Journal of
Robotics Research2011;:1–29.
Argall B.D., Chernova S., Veloso M., Browning B. A
survey of robot learning from demonstration. Robotics
and autonomous systems2009;57(5):469–83.
Asada M., Yoshikawa Y., Hosoda K. Learning by observation without three-dimensional reconstruction. Intelligent
Autonomous Systems (IAS)2000;:555–60.
Bentivegna D.C., Atkeson C.G., Ude A., Cheng G. Learning to act from observation and practice. International
Journal of Humanoid Robotics2004;1(04):585–611.
Bohg J., Morales A., Asfour T., Kragic D. Data-driven
grasp synthesis: A survey. Robotics, IEEE Transactions
on2014;30:289–309.
Buch A.G., Kraft D., Kämäräinen J.K., Petersen H.G.,
Krüger N. Pose estimation using local structure-specific
shape and appearance context. In: Robotics and Automation (ICRA), IEEE International Conference on. IEEE;
2013. p. 2080–7.
Burgard W., Cremers A.B., Fox D., Hähnel D., Lakemeyer G., Schulz D., Steiner W., Thrun S. Experiences
with an interactive museum tour-guide robot. Artificial
intelligence1999;114(1):3–55.
Chao C., Cakmak M., Thomaz A.L. Towards grounding
concepts for transfer in goal learning from demonstration.
In: Development and Learning (ICDL), IEEE International Conference on. IEEE; volume 2; 2011. p. 1–6.
Chen Y., Hsu C.W., Wah B.W. Sgplan: Subgoal partitioning and resolution in planning. Edelkamp et al(Edelkamp,
Hoffmann, Littman, & Younes)2004;.
Dantam N., Essa I., Stilman M. Linguistic transfer of
human assembly tasks to robots. In: Intelligent Robots

6. Conclusions
In this paper, a novel skill learning approach has
been proposed, which allows a robot to acquire
new skills for object manipulation by observing a
demonstration. VSL utilizes visual perception as
the main source of information. As opposed to
conventional trajectory-based learning approaches,
VSL is a goal-based robot learning approach that
focuses on achieving the desired goal configuration
of objects relative to one another while maintaining the sequence of operations. VSL is capable
of learning and generalizing multi-operation skills
from a single demonstration while requiring minimum a priori knowledge about the environment.
It has been shown that integrating VSL with a
conventional trajectory-based approach eliminates
the need to program the primitive actions manually and to tune them according to the desired
task. The robot learns the required primitive actions through kinesthetic teaching using Imitation
Learning. Then it learns the sequence of actions
to reach the desired goal of the task through VSL.
The obtained framework easily extends and adapts
robots capabilities to novel situations, even by users
without programming ability. It has been illustrated that to utilize the advantages of standard
symbolic planners, VSL’s original planner can be
replaced with an already available action-level symbolic planner. In order to ground the actions,
the symbolic actions are defined in the planner
and VSL maps identified preconditions and effects
in a formalism suitable to be used at the symbolic level. Experimental results have shown the
improvement in generalization to find a solution
that can be acquired from both multiple and single
22

and Systems (IROS), IEEE/RSJ International Conference
on. IEEE; 2012. p. 237–42.
Dantam N., Koine P., Stilman M. The motion grammar for
physical human-robot games. In: Robotics and Automation (ICRA), IEEE International Conference on. IEEE;
2011. p. 5463–9.
Ehrenmann M., Rogalla O., Zöllner R., Dillmann R.
Teaching service robots complex tasks: Programming by
demonstration for workshop and household environments.
In: Field and Service Robots (FSR), International Conference on. volume 1; 2001. p. 397–402.
Ekvall S., Kragic D. Robot learning from demonstration:
a task-level planning approach. International Journal of
Advanced Robotic Systems2008;5(3):223–34.
Feniello A., Dang H., Birchfield S. Program synthesis
by examples for object repositioning tasks. In: Intelligent Robots and Systems (IROS), IEEE/RSJ International Conference on. IEEE; 2014. p. 4428–35.
Golland D., Liang P., Klein D. A game-theoretic approach
to generating spatial descriptions. In: Empirical methods
in natural language processing, conference on. Association
for Computational Linguistics; 2010. p. 410–9.
Guadarrama S., Riano L., Golland D., Gouhring D.,
Jia Y., Klein D., Abbeel P., Darrell T. Grounding
spatial relations for human-robot interaction. In: Intelligent Robots and Systems (IROS), IEEE/RSJ International Conference on. IEEE; 2013. p. 1640–7.
Harnad S. The symbol grounding problem. Physica D:
Nonlinear Phenomena1990;42(1):335–46.
Ijspeert A.J., Nakanishi J., Hoffmann H., Pastor P.,
Schaal S. Dynamical movement primitives: learning attractor models for motor behaviors.
Neural
computation2013;25(2):328–73.
Ijspeert A.J., Nakanishi J., Schaal S. Trajectory formation for imitation with nonlinear dynamical systems. In:
Intelligent Robots and Systems (IROS), IEEE/RSJ International Conference on. IEEE; volume 2; 2001. p. 752–7.
Ikeuchi
K., Suehiro
T.
Toward an assembly plan
from observation. i. task recognition with polyhedral objects. Robotics and Automation, IEEE Transactions
on1994;10(3):368–85.
Jetchev N., Lang T., Toussaint M. Learning grounded
relational symbols from continuous data for abstract reasoning. 2013.
Kelleher J.D., Kruijff G.J.M., Costello F.J. Proximity in
context: an empirically grounded computational model of
proximity for processing topological spatial expressions.
In: Proceedings of the 21st International Conference on
Computational Linguistics and the 44th annual meeting
of the Association for Computational Linguistics. Association for Computational Linguistics; 2006. p. 745–52.
Konidaris G., Kaelbling L.P., Lozano-Perez T. Constructing symbolic representations for high-level planning.
In: Twenty-Eighth AAAI Conference on Artificial Intelligence. 2014. .
Kormushev P., Calinon S., Caldwell D.G. Robot motor
skill coordination with EM-based reinforcement learning.
In: Intelligent Robots and Systems (IROS), IEEE/RSJ
International Conference on. 2010. p. 3232–7.
Kroemer O., Peters J. A flexible hybrid framework for
modeling complex manipulation tasks. In: Robotics and
Automation (ICRA), IEEE International Conference on.
IEEE; 2011. p. 1856–61.
Kronander K., Billard A. Online learning of varying
stiffness through physical human-robot interaction. In:

Robotics and Automation (ICRA), IEEE International
Conference on. IEEE; 2012. p. 1842–9.
Krüger N., Piater J., Wörgötter F., Geib C., Petrick R.,
Steedman M., Ude A., Asfour T., Kraft D., Omrcen D.,
et al. A formal definition of object-action complexes and
examples at different levels of the processing hierarchy.
PACO-PLUS Technical Report2009;.
Kuniyoshi Y., Inaba M., Inoue H. Learning by watching:
Extracting reusable task knowledge from visual observation of human performance. Robotics and Automation,
IEEE Transactions on1994;10(6):799–822.
Lopes M., Santos-Victor J. Visual learning by imitation with motor representations. Systems, Man, and
Cybernetics, Part B: Cybernetics, IEEE Transactions
on2005;35(3):438–49.
Lowe
D.G.
Distinctive image features from scaleinvariant keypoints. International journal of computer
vision2004;60(2):91–110.
Mason M., Lopes M. Robot self-initiative and personalization by learning through repeated interactions. In:
Human-Robot Interaction (HRI), 6th ACM/IEEE International Conference on. IEEE; 2011. p. 433–40.
Meltzoff A.N., Moore M.K. Newborn infants imitate adult
facial gestures. Child development1983;:702–9.
Moratz R., Tenbrink T. Spatial reference in linguistic
human-robot interaction: Iterative, empirically supported
development of a model of projective relations. Spatial
cognition and computation2006;6(1):63–107.
Niekum S., Osentoski S., Konidaris G., Barto A.G.
Learning and generalization of complex tasks from unstructured demonstrations. In: Intelligent Robots and
Systems (IROS), IEEE/RSJ International Conference on.
IEEE; 2012. p. 5239–46.
Papazov C., Haddadin S., Parusel S., Krieger K., Burschka
D. Rigid 3D geometry matching for grasping of known
objects in cluttered scenes. The International Journal of
Robotics Research2012;31(4):538–53.
Pardowitz M., Zollner R., Dillmann R. Incremental acquisition of task knowledge applying heuristic relevance
estimation. In: Robotics and Automation (ICRA), IEEE
International Conference on. IEEE; 2006. p. 3011–6.
Park G., Ra S., Kim C., Song J. Imitation learning of robot
movement using evolutionary algorithm. In: Proceedings
of the 17th World Congress, International Federation of
Automatic Control (IFAC). 2008. p. 730–5.
Pastor P., Hoffmann H., Asfour T., Schaal S. Learning and generalization of motor skills by learning from
demonstration. In: Robotics and Automation (ICRA),
IEEE International Conference on. IEEE; 2009. p. 763–8.
Randell D.A., Cui Z., Cohn A.G. A spatial logic based
on regions and connection.Knowledge Representation and
Reasoning1992;92:165–76.
Rizzolatti G., Fadiga L., Gallese V., Fogassi L. Premotor
cortex and the recognition of motor actions. Cognitive
brain research1996;3(2):131–41.
Rusu R.B., Blodow N., Beetz M. Fast point feature histograms (FPFH) for 3D registration. In: Robotics and
Automation (ICRA), IEEE International Conference on.
IEEE; 2009. p. 3212–7.
Skubic M., Perzanowski D., Blisard S., Schultz A.,
Adams W., Bugajska M., Brock D. Spatial language
for human-robot dialogs. Systems, Man, and Cybernetics, Part C: Applications and Reviews, IEEE Transactions
on2004;34(2):154–67.
Steder B., Rusu R.B., Konolige K., Burgard W. NARF:

23

3D range image features for object recognition. In: Workshop on Defining and Solving Realistic Perception Problems in Personal Robotics at the IEEE/RSJ International
Conference on Intelligent Robots and Systems (IROS).
volume 44; 2010. .
Su Y., Wu Y., Lee K., Du Z., Demiris Y. Robust grasping for an under-actuated anthropomorphic hand under
object position uncertainty. In: Humanoid Robots (Humanoids), IEEE-RAS International Conference on. 2012.
p. 719–25.
Verma D., Rao R. Goal-based imitation as probabilistic
inference over graphical models. Advances in neural information processing systems2005;18:1393–400.
Video Video accompanying this paper, available online.
https://youtu.be/1xFeOZaMpK4; 2013a.
Video Video accompanying this paper, available online.
https://youtu.be/W6zgdve3e1A; 2013b.
Video Video accompanying this paper, available online.
https://youtu.be/baF9bCidcyw; 2015.
Yeasin M., Chaudhuri S. Toward automatic robot programming: learning human skill from visual data. Systems, Man, and Cybernetics, Part B: Cybernetics, IEEE
Transactions on2000;30(1):180–5.

24

