# Presentation: 496230

## Advanced Robotic Software Architecture

- Issa A.D. Nesnas, Ph.D.
- Jet Propulsion Laboratory
- February 6, 2002

## Presentation Overview

- Background and Overview
- Objectives
- Technology Development
- Vision-based Manipulation
- Vision-based Navigation
- CLARAty – Resuable Robotic Software
- Accomplishments
- Functional Layer Accomplishments - Examples on code reusability
- Decision  Layer Accomplishments – Multi-target planning scenario
- Future Plans
- Publications and website

- CLARAty - I.A.N.

## Vision-based Behaviors

## Objectives

- Autonomous manipulation from a rover platform
- Single and multiple rock sample acquisitions from a rover platform designated from ***one*** to ***two*** meters away
- Instrument placement on a target designatd from a distance of more than ***five*** meters away
- Autonomous navigation from a rover platform
- Vision-based obstacle detection and avoidance

- CLARAty - I.A.N.

## The Rocky  7  Rover

- The Rocky  **7**  Rover

- CLARAty - I.A.N.

## Vision Processing

- Elevation Map
- Interpolated Smoothing

- Select Target

- Filter and Rectify Images

- Range Map

- CLARAty - I.A.N.

## Visual Target Tracking

- Intensity threshold

- Max. rock height search

*Small rock tracking sequence*

- CLARAty - I.A.N.

## Multiple Target Tracking

- Body Camera View

- CLARAty - I.A.N.

## Target Grasping

- Select one or multiple targets from 1-2 meters away
- Visually track target
- Use rover arm to acquire samples
- Use rover sensing to ensure solid grasp of the rock

- CLARAty - I.A.N.

## Instrument Placement on Target

- Select target from 5 meters away
- Visually track target
- Final maneuver to adjust vehicle orientation relative to target

- Estimated target location and surface normal obtained from final stereo image acquisition
- Move mast instrument along target’s surface normal
- Stop servoing upon contact with target

- CLARAty - I.A.N.

## Vision-based Navigation on Rocky 8 

- GESTALT

- CLARAty - I.A.N.

## CLARAty – Resuable Robotic Software

## What is CLARAty?

- CLARAty is a unified and *reusable* framework that provides base functionality and aims  at facilitating the integration of new technologies on various rovers and robotic platforms

- CLARAty - I.A.N.

## High-level Requirements

- Capture robot-specific domain knowledge within the framework
- Provide Robotic Components that are:
- Reusable, easily reconfigurable, modular, extendible, mature, easy to use by non-experts, platform independent, testable, scalable, maintainable, efficient, easily integrated, well-defined, and well-documented

- CLARAty - I.A.N.

## A Two-Layered Architecture

**        **CLARAty = Coupled Layer Architecture for Robotic Autonomy

- CLARAty - I.A.N.

## Currently Supported Platforms

- Solaris

- Motorola 68K

- Intel x86

- Ames

- JPL

- CMU

***Rocky 8***

- PowerPC

- VxWorks

- VxWorks

- Linux

***Rocky 7***

***K9***

- JPL

***ROAMS***

- JPL

- Linux

- Intel x86

- CLARAty - I.A.N.

## Adapting to a Rover

- Hardware Drivers

- Generic
- Functional Layer

- Rocky 8
- Specialized
- Classes
- &
- Objects

- Connector

- Decision Layer
- (e.g. CASPER/TDL)

- Multi-level access Connector

- Rocky 8
- Models/
- Heuristics

- CLARAty - I.A.N.

## The Functional Layer

- CLARAty - I.A.N.

## Decision Layer- Planning & Execution

- Goal discarded due to memory resource conflict

- Unknown obstacles causes obstructed path. Science targets are re-sequenced

- Goal discarded due to energy resource conflict

- Goal initially discarded due to  memory resource conflict

- CLARAty - I.A.N.

## Generic Algorithms

**Wheeled Locomotor Hierarchy – works for Rocky 8, Rocky 7, Fido, K9, and much more**

- CLARAty - I.A.N.

## Rover Mobility

- CLARAty - I.A.N.

## R8 Specific Rover Implementation

- Mast

- LeggedLoc

- Wheeled Locomotor

- Locomotor

- CoordMotionSystem

- Motor

- BBMotor

- ControlledMotor

- Analog_IO

- Digital_IO

- IO

- Manipulator

- Arm

- R8_Arm

- R8_Mast

- Implements general fwd &
- inv. kinematics & joint ctrl

- Specialized inv. Kinematics (overrides default)
- Attaches proper motors
- Attaches proper cameras for mast
- Adds filter wheel

- R8_Locomotor

- Attaches proper motors
- Restricts Steering to 2 wheels

- R8_Motor

- R8_Rover

- R8_Arm

- R8_Mast

- R8_Locomotor

***R8***

- Widget Board

- Widget AIO

- Widget DIO

- Widget Motor

- HCTL 1100 Chip

- Non reusable Code

- Reusable Code

- Trajectory

- Trajectory_Generator

- Timers

- CLARAty - I.A.N.

## R7 Specific Rover Implementation

- Mast

- WheeledLoc

- LeggedLoc

- RBLoc

- Locomotor

- CoordMotionSystem

- Motor

- BBMotor

- ControlledMotor

- Analog_IO

- Digital_IO

- IO

- Manipulator

- Arm

- R7_Arm

- R7_Mast

- Implements general fwd &
- inv. kinematics & joint ctrl

- Specialized inv. Kinematics (overrides default)
- Attaches proper motors
- Attaches proper cameras for mast
- Adds filter wheel

- R7_Locomotor

- Attaches proper motors
- Restricts Steering to 2 wheels

- LM629Chip

- LM629Motor

- R7_Rover

- R7_Arm

- R7_Mast

- R7_Locomotor

***R7***

- Device Drivers

- VPAR10Board

- Non reusable Code

- Reusable Code

- CLARAty - I.A.N.

## Code Resuability

**Wheeled Locomotor Hierarchy – worked for Rocky 8, Rocky 7, Fido, K9, and much more**

- CLARAty - I.A.N.

## FY01 CLARAty Members & Collaborators

- Jet Propulsion Laboratory (CLARAty/Rocky8 Team)
- Issa A.D. Nesnas, Task Manager, (34)
- Max Bajracharya (34)
- Hari Das (34)
- Tara Estlin (36)
- Richard Petras (34)
- Edward Barlow (34)
- Caroline Chouinard (36)
- Dan Helmick (34)
- Stanley Lippman (Consultant)
- Darren Mutz (36)
- Ashitey Trebi-Ollennu (35)
- Paolo Pirjanian (35)
- Kevin Watson (34)
- Matt Robinson (35)
- Gene Chalfant (34)
- Meemong Lee (38)
- Ames Research Center
- Anne Wright
- Clay Kunz
- Esfandiar Bandari
- Maria Bualat
- Richard Washington

- Carnegie Mellon University (FL/DL)
- Reid Simmons
- Chris Urmson
- Kam Lasater
- David Wettergreen
- CLEaR Team (collaborator)
- Forest Fisher, Task Manager, (36)
- Tara Estlin (36)
- Dan Gaines (36)
- Steve Schaffer (36)
- Caroline Chouinard (36)
- MER Team (collaborator)
- Mark Maimone
- ROAMS Team (collaborator)
- Abhi Jain
- Robert Steele
- Jonathan Cameron
- Garett Sohl

- CLARAty - I.A.N.

## Video Presentation

## Summary

- CLARAty is a collaborative effort of several robotic  researchers.
- Provides a repository of reusable software components used by JPL, Ames, & CMU. Over 25 users and developers
- Attempts at capturing well-known robot technologies in a basic framework for researchers
- Allows researchers to integrate novel technologies at different levels of the architecture
- Runs various releases of the packages on different rovers.
- Integrates with simulation
- Runs multiple heterogeneous robots

- CLARAty - I.A.N.

## Future Plans

- Improve current software infra-structure
- Complete development of several packages
- Fully document all packages and technologies
- Provide navigation infra-structure to support and compare  different navigation technologies
- Extend packages to fully support other rovers and mobile platforms
- Enhance Manipulation package and adapt to Rocky 8 & Rocky 7
- Provide Basic Vision and Stereo Vision Packages using JPL and Ames Stereo Technology
- Provide simulation components at various levels of granularity
- Provide automated means for accessing Functional Layer from Decision Layer.
- Provide richer functionality to the Decision Layer
- Provide resource queries at various levels

- CLARAty - I.A.N.

## Publications & Website

**http://claraty.jpl.nasa.gov**
- I.A.D. Nesnas, R. Volpe, T. Estlin, H. Das, R. Petras D. Mutz, "Toward Developing Reusable Software Components for Robotic Applications" *Proceedings of the International Conference on Intelligent Robots and Systems (IROS),* Maui Hawaii, Oct. 29 - Nov. 3 2001. pdf (8 pages, 2MB)
- T. Estlin, R. Volpe, I.A.D. Nesnas, D. Mutz, F. Fisher, B. Engelhardt, S. Chien, "Decision-Making in a Robotic Architecture for Autonomy." *Proceedings of 6**th** International Symposium on Artificial Intelligence, Robotics, and Automation in Space (i-SAIRAS),* Montreal Canada, June 18-21 2001. pdf (8 pages, 72KB)
- R. Volpe, I.A.D. Nesnas, T. Estlin, D. Mutz, R. Petras, H. Das, "The CLARAty Architecture for Robotic Autonomy." *Proceedings of the 2001 IEEE Aerospace Conference, Big Sky Montana, March 10-17 2001.* pdf (12 pages, 470 KB)
- R. Volpe, I.A.D. Nesnas, T. Estlin, D. Mutz, R. Petras, H. Das, "CLARAty: Coupled Layer Architecture for Robotic Autonomy." *JPL Technical Report D-19975*, Dec 2000. pdf (116 pages, 904 KB)

- CLARAty - I.A.N.