------------------------------------ ----------------------------------
  **Jet Propulsion Laboratory**        White Paper 5/12/2004

  ------------------------------------ ----------------------------------

Humanoids for Urban Operations

*Fostering Robo Urbanus*

Dr. Adrian Stoica

Jet Propulsion Laboratory, 4800 Oak Grove Drive, Pasadena, CA 91109

Tel: (818) 354-2190, Email: <adrian.stoica@jpl.nasa.gov>

# Executive Summary

Humanoids can offer significant tactical advantages in urban warfare,
peacekeeping missions, or the war on terror, increasingly important
aspects of our defense operations. Reducing the risk of placing soldiers
in harm's way, the humanoid shape of this *robo urbanus* is more
suitable than other forms for operating in environments built for
humans, combining in the same platform various capabilities needed
during operations, e.g. climbing a ladder, going down in a rocky cave,
carrying a human in its arms, being transportable in a regular car seat,
or more efficiently and with higher acceptance interacting with
civilians in war zone. These characteristics would allow them to perform
a range of operations, such as scouting in streets, buildings, or
underground shelters, rescue missions carrying out prisoners and injured
personnel, using ordinary military equipment; all these are things that
other robotic platforms currently developed can't do. However, although
mobility, flexibility and human environment adaptation offered by human
shape is a convenient advantage, ***the key reason for preferring
humanoids is their optimal shape for being taught by humans and learning
from humans,* considered the only ways to develop cognitive and
perceptual/motor skills for truly intelligent, cognitive robots**. We
propose a research effort toward demonstrating *robo urbanus,* which
starts with a near-term objective of developing basic cognitive and
motor skills of a humanoid to a level at which it can manipulate objects
and move freely in a building environment *in an autonomous, self-aware
way*. We propose a developmental approach, gradually enhancing robot
capabilities through guided human interaction/fostering. Initially
endowing the robots with primitive needs, instincts and reflexive
behaviors, we would create capabilities to "dream" (imagine/simulate
potential solutions), "play" (test/validate solutions in real-world),
and "imitate" (follow guidance of a human). A key characteristic of our
approach is the emphasis on *teaching/fostering,* a component overlooked
by other approaches, yet at least as necessary as learning.[^1] As in
nature, even the greatest learning ability does not make for the absence
of a teacher from whom to learn. Teaching is performed initially by
humans, who select appropriate means/lessons/examples; consequently the
*robots themselves exercise teaching,* *as proof of learning* and
"understanding" what was learned, of proving awareness of self and
others, exercising internal models, showing imagination and adaptation
to new situations. We also enable robots to "doubt" as an indication of
exercising internal reasoning (*"dubito ergo cogito"*), i.e. analyze
inputs from the outside world, match with existing models and react by
acceptance/ rejection and model updates. A HOAP-2 Fujitsu humanoid is
used for demonstrating the approach in a reduced scale habitat; a
full-scale humanoid development with cognitive and operational
capabilities and demonstrations with teams of cooperating humanoids are
follow-ons. While the focus is on developing humanoids, resulting
fostering techniques will benefit cognitive development of other robots
as well*.*

[The Vision]{.underline}

# Robo urbanus: offering significant tactical advantages in urban warfare and peacekeeping missions

Recent events throughout the world stress the need to improve our
superiority in urban warfare, peacekeeping missions and operations
characteristics to the war on terror. On one hand intelligent autonomous
"extension" robots, of various shapes, with sensing/motor/cognitive
capabilities different than ours - extension of ours - will provide
invaluable service to the forces, offering advantages in sensing,
communication and power projection, such as robotics flying insects
entering buildings through the windows, UAVs with firing power
controlling from above, and unmanned vehicles controlling the street. On
the other hand, "replacement" robots - substitutes for humans in what
humans do best - would eliminate the risk of placing soldiers in harm's
way. The humanoid shape of this "*robo urbanus"* is more suitable than
other for operating in environments built for humans, combining in the
same platform the capabilities needed to climb a ladder, go down in a
rocky cave, carry a human in the arms, travel in a regular car seat, or
to be accepted by civilians for interactions/communication in war zones.
These characteristics would allow humanoids perform a range of
operations including: scouting in streets, buildings or underground
shelters, rescue missions in which they would carry out prisoners and
injured personnel, freeing roads from objects/mines placed in front of
vehicles, ad-hoc assembly tasks in dangerous areas, manipulation and
assembly in hazardous built-for-humans environments including under
nuclear, chemical and biological attacks, (un) packing, interacting with
civilians in enemy territory and other things (including use of current
military equipment) that current robots can't do. Acceptance by own
forces as companion is interestingly formulated by the following remark
from an Army officer "If I can't carry it and it can't carry me I don't
want it".

Several advantages of humanoids are summarized here after \[1\]: a)
Human interaction with robots is easier if the robots are humanoid; b)
Robot acceptance by humans is easier for humanoid shape; c) Efficiency
of teaching/programming a robot is highest with humanoids. In particular
related to this last point, one should stress here that while the
anthropomorphic shape positions them better than other robotic systems
to operate in environments (and with tools) built for humans[, the most
important advantage is that it offers them the *optimal shape for being
taught by humans and learning from humans (developed/fostered by
humans),* which are the enabling factors for advancements in robot
cognitive development, motor skills, and autonomy.]{.underline} A wealth
of knowledge ready to be transmitted to humanoids is waiting for being
used: while in the first stage robots may learn directly from humans,
they could in the future learn by watching humans on training videos and
commercial movies. This white paper addresses the robot development,
fostering, teaching and learning, and focuses on demonstrations on
humanoids. One believes that while providing a unique capability for
humanoids, the developmental approach and teaching/fostering techiniques
developed part of the proposed task will greatly benefit other robots as
well, playing a key role in achieving intelligent, cognitive robots in
the future.

[How to get there]{.underline}

While the above scenario may appear futuristic, Japanese humanoids prove
human resemblance in shape, size, and basic mobility already exists.
These robots can walk alone, go on stairs, and can be tele-operated to
handle various objects. However, the mechanical aspects of these robots
are more advanced than their processing capabilities -- while they can
give the visual appearance of human shape and motion, their cognitive
capabilities are practically absent. On the other hand, results of other
research worldwide can be integrated to the humanoid body to provide a
powerful platform to develop it to operational levels. Several
technologies available, yet not incorporated in humanoids include:
[language technologies,]{.underline} including voice recognition, speech
to text and voice synthesis, which are sufficiently developed to allow
simple interaction with the robot in spoken English (simple Japanese
exists in some research robots); f[ace/gesture recognition,]{.underline}
sufficiently developed to allow robots to read "moods" of instructor,
follow cues, etc (as prototyped e.g. on some MIT robots); [knowledge
base, dialog and logical reasoning]{.underline}, as illustrated e.g. by
Cyc, proving useful artificial intelligence; [improved vision, hearing,
olfaction, tactile, and other sensing,]{.underline} developed to a
certain extent and incorporated in various commercial devices
(artificial retinas, e-nose, e-tongue). Some of the
capabilities/technologies still needed include efficient and
human-friendly means to transfer cognitive and motor skills to robots,
cognition and self-awareness, perception from big sensory arrays (e.g.
skin) and an integrated platform that combines available technologies.

Incorporating available technologies and developing new ones on the same
integrated platform is an efficient way to bridge the gap between
current state of the art and *robo urbanus.* There is a tight connection
between achieving human-friendly means for cognitive/motor skill
transfer and interaction through dialog in natural language; similarly,
cognition and self-awareness are also related to development of
perceptual maps and schemes; embodiment and experimenting the world are
dependent on perceiving the world with multiple sensors, etc. We propose
to follow this path, in a developmental approach to provide the robot
with cognitive/motor capabilities.

A key distinguishing characteristic of our approach compared to other is
the emphasis on fostering. Our key beliefs for a successful path to
*robo urbanus* are:

1.  [**The essence of endowing robots with intelligence is development,
    not programming.**]{.underline} Development allows building of
    perceptions, schema, representations, and behaviors directly through
    interaction with the real world environment (a set of
    innate/pre-programmed capabilities is assumed). This is a gradual
    building proces, using previously learned categories. It allows the
    developer to better understand limitations of the operation and to
    design lessons.

2.  [**The key to development is robot fostering/teaching, and not robot
    learning**.]{.underline} It may be not the human capability to
    learn, but to teach that contributed greatly to our progress. While
    we will pay great attention to learning algorithms for the robot,
    and incorporate the best learning techniques available, in various
    flavors of unsupervised, reinforcement and supervised mode, our
    approach emphasizes the importance of fostering/teaching
    techniques[^2], largely overlooked by other approaches yet
    considered key to development of cognitive/motor skills. As
    examples, human imitation of the robot in its initial actions
    (before the robot itself starts to imitate), providing
    experiments/lessons of increasing difficulty and helping the robot
    ("keeping it by the hand") while learning. [^3]

3.  **[The main techniques for fostering/teaching by a human or robot
    are imitation, explanation, and demonstration.]{.underline}** The
    robot needs help during learning. In initial phase human imitation
    of robot movements provides the robot with feedback. Later its own
    imitation of the human helps acquiring new behaviors. Explanation is
    paramount for guidance and for understanding the
    movements/tasks/behaviors. Demonstration provides a solution on how
    to solve a problem. Direct help from the human, in the form of
    supporting the robot during its first steps, providing a helping
    hand in need, positioning it by hand, etc., all are a great help to
    the robot. Interactive teaching is exptremely important since it
    adapts to context.

4.  **[Robot's ability to teach is the proof of learning]{.underline}.**
    The ability to teach is a validation that the essence of the task is
    grasped, that it is generalized and can be applied in a different
    context, that it is "conscious", meaning it has a flexible
    representation in context of self and outside world, and a rationale
    for why it is that way. With humans it is also common to say that
    professors really learn a subject only when/after they teach it.

![](media/image1.jpeg){width="1.7347222222222223in"
height="1.4979166666666666in"}![](media/image2.jpeg){width="1.5173611111111112in"
height="1.5027777777777778in"}
![](media/image3.jpeg){width="1.2270833333333333in"
height="1.5034722222222223in"}![](media/image4.jpeg){width="1.3479166666666667in"
height="1.5041666666666667in"}

Fig 1. Human imitating the robot, before the robots imitates human.
Teaching the robot walk, later providing only "a helping hand". Pictures
in our JPL lab, with a HOAP-2 robot.

**On embodiment and self-awareness.** We subscribe the view that
associates intelligence, concepts and representations of a robot with
its body, as the context in which awareness of the body, internal models
and reasoning should develop [^4]. We will also explore the idea of
making robots "doubt" as an indication of exercising internal reasoning
(*"dubito ergo cogito"*), by analyzing inputs from the outside world,
matching them with an existing model of the world, and reacting, by
acceptance or doubt/rejection, with model updates.

Our previous work demonstrates the capability of transferring motor
skills to anthropomorphic robots through vision-based imitation. After
initially the humans imitated the robot arm movements flailing at random
at first, the robot watching the human developed an association between
its motor commands and visual inputs as reactions to its moves.
Reversly, it later commanded its arm to positions associated to arm
movements of instructor, imitating and learning from its moves.
Experiments with two robots imitating and learning from each other were
also performed. We will leverage on this work employing neuro-fuzzy
structures fusing vision, language and information from other sensors.

A 50cm tall Humanoid Open Architecture Platform, 2^nd^ generation,
(HOAP) Fujitsu robot will be used as initial platform (see Fig. 1). It
has the dimensions of an infant, 29 motors controlling joints of arm,
various sensors including 2 cameras of 640x480 resolution, gyros,
pressure sensors, etc; it can walk on a smooth surface. It can operate
under own computer or under wireless control by human acting via command
environment under real-time Linux. The potential of this robot
(including on climbing stairs, etc) is illustrated in a series of movies
available on-line (the software for the applications in the movie was
not provided with the robot)
<http://www.automation.fujitsu.com/products/products092.html>

[Proposed Task]{.underline}

**[Objectives:]{.underline}** **Develop cognitive, awareness and motor
capabilities for a humanoid robot through fostering and teaching
techinques; demonstrate it by walking and manipulating objects
autonomously and self-aware.** Prove it first on a HOAP-2 robot able to
navigate a real building environment (of its size) including stairs, and
doors; select, pick-up and transport/place objects, perform simple
operations related to construction assembly (select/lift/transport/place
in right position). As follow-on demonstrate cooperative assembly by 2
or more humanoid robots and develop a full-scale humanoid on which
demonstrate the same cognitive and motor capabilies.

**[Benefits:]{.underline}** From an operational perspective the
capabilities developed in this effort provide a demonstration of
autonomous humanoid navigation and locomotion in buildings, object
handling/ manipulation/asssembly, as a proof of concept for the
potential of a humanoid operating in an urban area. From an intelligent
systems perspective, fostering and teaching techniques are highly useful
in developing cognition and awareness, critical for bulding trully
intelligent robots.

**[Tasks, planning and schedule (assuming a level of effort of 3 FTE,
approx \$50k/month):]{.underline}**

1.  Enhance sensing capability (2 months) . End: 2 months after Contract
    Start Date (CSD +2)

    a.  Add extra sensors and sensor-processing HW/SW.

        i.  Pressure/proximity sensors will be added, mostly on
            arms/hands and legs/feet.

        ii. Add simple data acquisition HW/SW

    b.  Add image processing functions (machine vision SW). Incorporate
        software used on other JPL robots, for object detection
        recognition in images from 640x480 cameras.

2.  Prepare set-up for developmental experiments (1 month). Ends: CSD+3

    a.  Develop secure area and watchdogs preventing robot from unsafe
        moves.

    b.  Baby walker - with measurement of force exerted on the
        supporting rail by the robot.

    c.  Security cord -- with measurement of force exerted on the cord
        by the robot.

    d.  Development simple optimization process minimizing the force
        exerted on the rail/cord, such that the robot eventually exerts
        no force (e,g walks alone without support).

3.  Define and implement the innate functionality (2 months) Ends: CSD+5

    a.  Main needs, a pain/pleasure function of sensorial input -- as
        cost functions for main routine -- a continuous optimization
        algorithm, satisfying the above cost functions

    b.  Reflexive behaviors

    c.  Optimization mechanism/operating system

4.  Installation of Language Tools, testing, choice of vocabulary (1
    month) Ends: CSD+6

    a.  Speech recognition SW (e.g. IBM Via Voice for Linux)

    b.  Text-to-speech SW (e.g. ViaVoice speech-to-text)

    c.  Feedback loops, dreams and play, robot talking and listening to
        its talk

5.  Development of perceptual/cognitive maps (3 months) Ends: CSD+9

    a.  Robot flailing, moving all joints and recording sensor
        measurements. Building associations between motor commands and
        limb/body postures through sensor readings

    b.  Perceptual cognitive associative maps for self, humans, world.
        Dreams and self-play.

    c.  Learning to associate commands to limbs, simple motor behaviors,
        perception of space with language/words -- being able both to
        react to commands like "faster", "lift your right leg higher",
        and express its own actions: "raise left hand to eye level",
        "bend the elbow, turn half right", "look forward". Dreams and
        play.

6.  Develop simple visuo-motor coordination of arm/hand. (4 months)
    Ends: CSD+13

    a.  Reaching (eye-hand and eye-arm coordination)

    b.  Grabbing, catching an object

    c.  Learning of motor sequences/patterns (e.g. hammering) correlated
        with a task.

    d.  Teaching by the robot back to the human (expression of
        "understanding") -- feedback from human action

7.  Develop various kinds of gait/balance on various surfaces, carry
    objects. 5 mths Ends: CSD+18

    a.  Use pressure minimization optimization algorithms on walking
        support to gradually walk freely without support through
        exercising various sets of legs/body commands.

    b.  Introduce disturbances to enable the robot to walk on various
        surfaces, slighlty inclined

    c.  Use vision and human aid to learn to navigate stairs and walk
        over obstacles

8.  Construction/assembly related tasks

    a.  Walk carrying an object

    b.  Push/drag an object

    c.  Plan and execute walking between references indicated by
        instructor

    d.  Learn to balance body/recover from falling, raising-up using
        arms

[FOLLOW-ON OPTIONS:]{.underline}

9.  Cooperative assembly: coordinated actions of 2+ robots (6 months)
    CSD+ 24

    a.  Acquire another HOAP-2 robot(s) and transfer some of primitive
        skills

    b.  Port directly some of the software, build more cognitive maps
        related to robot mate

    c.  Practice/automate the "teaching" by the first robot to the
        second robot for all actions learned by first robot from human.
        Second robot practices in turn teaching the first one.

10. Scale-up to full scale autonomous humanoid (6-mth) \~\$400k
    procurements/contracts CSD+30

    a.  Use one avaliable JPL arm (make second), Harada dexterous hands,
        more sensors

    b.  Use space/mil components

    c.  Transfer all computing capability (now on outside laptop) to the
        robot

11. Port capabilities from HOAP-2 (2 months) End: CSD+36

12. Learn assembly/construction operations (4 months) End: CSD+40

    a.  Learn to use new tools

    b.  Learn to read simple schematics

**[Schedule and Milestones:]{.underline}**

**Phase 1:** (Assumes CSD June 1, 2004) 06/2004-10/2004 (4 months) CSD+4

In this period, we will get additional 6 summer students (seniors,
complementary skills in programming, mechanics, control, signal
processing, mathematics) and one NASA Faculty Professor (Prof. Chris
Tseng, project coordination, intelligent robots/controls,
mathematics/mechanics) working on this project in our lab. Complete
tasks 1,2, 3a, 4.

**Phase 2:** 10/2004 --12/2005 (14 months) (CSD+18)

In this period we would have achieved cognitive/motors skills and
self-awareness developed through teaching/learning and demonstrated by
HOAP-2 robot autonomously navigating/handling objects under verbal
guidance of instructor. Complete tasks 5-8

**Phase 3:** (Follow-on options). Dates and Capability achieved at the
end

3.1: 12/2005- 06/2006 (CSD+24) Cooperative HOAP-2 robots

3.2: 06/2006- 12/2006 (CSD+30) Full scale humanoid body/smarts

3.3: 12/2006- 06/2007 (CSD+36) Full scale humanoid with
cognitive/motor/skills/awareness

3.4 06/2007-10/2007 (CSD+40) Humanoid ready for simple
construction/assembly work

[The Team]{.underline}

# Key Contributors: Credentials, areas of expertise/contribution http://ehw.jpl.nasa.gov/humanoid

**Dr. Adrian Stoica**, **(Principal Investigator)** PhD in Humanoid
Robots, VUT, Australia, 1995 with the thesis: "Motion learning by robot
apprentices". Led several multi-million dollar projects for DARPA, USAF,
NASA; 18 years of R&D in learning/adaptive hardware;100+ publications;
key contributions to evolvable hardware and learning systems; pioneer of
applying learning by imitation;. Graduate of 1st International Summer
Institute in Cognitive Sciences, Buffalo NY, 1994; Lew Allen Award
(highest JPL distinction for research) Position paper: A. Stoica, "Robot
Fostering Techniques for Sensory-Motor Development of Humanoid Robots"
*Journal of Robotics and Autonomous Systems. (Special Issue on Humanoid
Robots).* 37(2-3) 2001, Elsevier.

**Dr. Didier Keymeulen**, PhD in Robotics, Vrije Univ of Bruxells,
Belgium 1995, with the thesis: "A Dynamics Approach to Mobile Robot
Motion: the Stream Field Method". Experience in navigation, evolutionary
robotics/robot learning, fault-tolerant robotics. Selected paper: D.
Keymeulen, *et al*. On-line evolution for a self-adapting robotic
navigation system using evolvable hardware. *Artificial Life* *(Special
Issue on Evolutionary Robotics)* 4(4):359-393, 1999, MIT Press.

**Dr. Dan Mandutianu**, PhD in Artificial Intelligence, Technical
University of Bucharest, 1986, with the thesis "Intelligent robot
planning." Experience in Knowledge representation, reasoning, SW
architecture, rule-based systems, AI. Selected paper: D. Mandutianu and
S. Voinea, Robots: Skill and Sensitive Behaviour, In I. Plander (Ed)
*Artificial Intelligence and Information-Control Systems of Robots*, pp
229-232, 1994, North-Holland, Amsterdam

**Dr. Chris Assad**, PhD in Electrical Engineering, Caltech, 1997, with
the thesis: "Electric field maps and boundary element simulations of
electrolocation in weakly electric fish." Experience*:* biomorphic
robotics, cerebellum, neuroethology , neural networks, computational
neuroscience, Position paper: C. Assad, M. Hartmann, M.G. Paulin (2002)
Control of a simulated arm using a novel combination of cerebellar
learning mechanisms *Neurocomputing* **44-46**C:273-281, 2002

**Prof. Chris Tseng,** PhD in Robotic and System Control, University of
Illinois at Urbana-Champaign, 1988 with the thesis: "Integral Manifold
in Nonlinear systems and Robotics". He has 20 years of R&D experience in
robotics, control, and artificial intelligence. He is a Professor and
Director of Computational Intelligence Lab. in Computer Science
Department of San Jose State University. His industry experience ranges
from being a consultant, a manager, to a VP of start-up and public
hi-tech companies. Before joining San Jose State University, he taught
at Santa Clara University and Duke University. Dr. Tseng was a David
Packard Fellow during 1988-1990. Position paper: H. Chris Tseng, \"Fuzzy
robot controller with time-scale decomposition,\" *ICARCV\'92 Second Int
Conf. on Automation, Robotics and Computer Vision*, Singapore, 1992

[^1]: A. Stoica, Robot Fostering Techniques for Sensory-Motor
    Development of Humanoid Robots" *Journal of Robotics and Autonomous
    Systems 37 (2001) 127-143 (attached)*

[^2]: In the animal world fostering is considered an important component
    to ensure survival of the species. Interestingly, it is been
    observed that the more "advanced" a species is, the longer the
    period of immaturity of its offspring --- in other words the longer
    the parents need to foster their children. It is this period when
    the young ones develop the skills that would make them successful in
    life.

[^3]: The parents act as first teachers taking the young ones through
    various phases of learning. In time the grown-up will in turn teach
    others (not seldom themselves learning more during/through teaching)

[^4]: Johnson and Lakoff argue that the shape of our bodies is critical
    to the representations we develop and use for both our internal
    thought and our language. If we are to build a robot with human-like
    intelligence, then it must have a human-like body in order to be
    able to develop similar sorts of representations. Can non-human
    entities develop human-like intelligence? Physical bodies may not be
    essential for artificial intelligence, but they would at least be
    convenient. Certainly the more humanoid the robot, the easier it
    would be to give it useful human behaviors.