# Presentation: 495208

## CLARAty: Towards Standardized Abstractions and Interfaces for Robotics Systems Coupled Layer Architecture for Robotic Autonomy

- Issa A.D. Nesnas
- Jet Propulsion Laboratory, California Institute of Technology
- In Collaboration with
- Ames Research Center
-  Carnegie Mellon University
- University of Minnesota
- JTARS Meeting May 11-12, 2005, Houston, Texas

## Motivation

## NASA/JPL Develops Various Rovers

- For research & flight

- ICRA2005 - I.A.N.

- April 18, 2005

## Would like to support ...

- ICRA2005 - I.A.N.

- April 18, 2005

## Problem and Approach

- Problem:
- Difficult to share software/algorithms across systems
- Different hardware/software infrastructure
- No standard protocols and APIs
- No flexible code base of robotic capabilities
- Objectives
- Unify robotic infrastructure and framework
- Capture and integrate legacy algorithms
- Simplify integration of new  technology
- Operate heterogeneous robots

- ICRA2005 - I.A.N.

- April 18, 2005

## Challenges in Interoperability

- Mechanisms and Sensors
- Hardware Architecture

## Different Mobility Mechanisms

- with different sensors

- From wheeled Rocker-
- bogies with different
- steering

- From three wheelers

- To inflatable
- wheels

- To wheels on
- articulatedlinks

- From wheeled to legged

- To four, six and even eight

- ICRA2005 - I.A.N.

- April 18, 2005

## For Example: Wheeled Locomotion

**Rocky 7 **

**Rocky 8 **

- ICRA2005 - I.A.N.

- April 18, 2005

## Reusable Wheeled Locomotion Algorithms

- General flat terrain algorithms and specialized full DOF algorithms

- ICRA2005 - I.A.N.

- April 18, 2005

## Manipulators and Sensor Suites

- 4 DOF Mast

- Given different capabilities, how much reuse can be achieved?

- ICRA2005 - I.A.N.

- April 18, 2005

## Challenges in Interoperability

- Mechanisms and Sensors
- Hardware Architecture

## Centralized Hardware Architecture

***FIDO***

- Actuator/Encoders

- Potentiometers

- PID Control in Software

- Video Switcher

- IMU

- RS232
- Serial

**PC104+**
- x86 Arch
- Framegrabbers
- Digital I/O
- Analog I/O
- Wireless Ethernet

- ICRA2005 - I.A.N.

- April 18, 2005

## Semi-centralized Hardware Architecture

**Compact PCI**
- PPC 750 Arch
- Framegrabbers
- Digital I/O
- Analog I/O
- Wireless Ethernet

***Rocky 7***

- Actuator/Encoders

- Potentiometers

- Parallel Custom Interface
- MUX/Handshaking

- Video Switcher

- Gyros

- Accels

- AIO

- PID Controllers

- ICRA2005 - I.A.N.

- April 18, 2005

## Semi-distributed Hardware Architecture

**Compact PCI**
- - x86 Arch
-  Wireless E/net
-  1394 FireWire
- - I2C Bus

- Actuator/Encoders

- Potentio-
- meters

- I2C

- 1394 Bus

- IMU

- Sun Sensor

***Rocky Widgets***
- Single-axis controllers
- Current sensing
- Digital I/O
- Analog I/O

- RS232

- Distributed Motion
- Control and Vision

***Rocky 8***

- ICRA2005 - I.A.N.

- April 18, 2005

## So what do we do?

## One Approach

- Use the best attributes from each system
-  and build a common platform

- OR
- Develop a model to deal with the variability

- Unfortunately this is not always possible

- ICRA2005 - I.A.N.

- April 18, 2005

## One Approach

- Use abstractions

- Interface at different levels

- Separate hierarchies

- Adapt as necessary

- ICRA2005 - I.A.N.

- April 18, 2005

## One Approach

- Develop
- Common data structures
- Physical & Functional Abstractions
-  E.g. motor, camera, locomotor. Stereo processor, visual tracker
- Unified models for the mechanism
- Putting it together
- Start with top level goals
- Elaborate to fine sub-goals
- Choose the appropriate level to stop elaboration
- Interface with abstractions
- Abstractions translate goals to action
- Specialize abstractions to talk to hardware
- Hardware controls the systems and provide feedback

- ICRA2005 - I.A.N.

- April 18, 2005

## A Two-Layered Architecture

_***THE DECISION LAYER:***_ Declarative model-based Global planning

**        **CLARAty = Coupled Layer Architecture for Robotic Autonomy

***INTERFACE:*** Access to various levelsCommanding and updates

_***THE FUNCTIONAL LAYER:***__*** ***_Object-oriented abstractionsAutonomous behaviorBasic system functionality

**A Two-Layered Architecture**

- Adaptation to a system

- ICRA2005 - I.A.N.

- April 18, 2005

## The Decision Layer

- Executives
- (e.g. TDL)

- General Planners
- (e.g. CASPER)

- Activity Database

- Rover Models

- FL Interface

- Plans

- ICRA2005 - I.A.N.

- April 18, 2005

## The Functional Layer

*<footer>*

*<date/time>*

- Transforms

- Motion Control

- Vision

- Estimation

- Input/Output

- Manipulation

- Navigation

- Communication

- Math

- Hardware Drivers

- Locomotion

- Rover

- Behaviors

- Path Planning

- Rocky 8

- Rocky 7

- K9

- FIDO

- Science

- Simulation

- Sensor

- Adaptations

## Standardizing Base Abstractions

- ICRA2005 - I.A.N.

- April 18, 2005

## Unify Mechanism Model

- ICRA2005 - I.A.N.

- April 18, 2005

## Putting it All Together

## Navigation Example - Swapping Algorithms

- Stereo Camera

- Stereo Engine

- <<active>>

***Navigator***

- Gestalt Navigator

- Grid Mapper

***Mapper***

- JPL Stereo

***Terrain Sensor***

- Locomotor

- R8_Locomotor

- Global Cost Func

- D* Path Planner

- Stereo Processor

- Camera R

- Camera L

- Synchronous/or
- Asynchronous
- e.g. Rate Set at: 10Hz
- used by other activities

- Asynchronous

- <<active>>

- Asynchronous
- e.g. Rate Set at: 5 Hz

- <<active>>

- Asynchronous
- e.g. Rate Set at: 8 Hz

- EKF Pose Estimator

- Pose Estimator

- Path Information

- Path Planner

- K9 Rover

***Rover***

- Decision Layer

- Commanding and State Updates

- ICRA2005 - I.A.N.

- April 18, 2005

## Navigation Example - Swapping Algorithms

- Stereo Camera

- Stereo Engine

- <<active>>

***Navigator***

- R7/Soj Navigator

***Mapper***

- JPL Stereo

***Terrain Sensor***

- Locomotor

- ROAMS_Locomotor

- Tangent Graph

- Stereo Processor

- Camera R

- Camera L

- Synchronous/or
- Asynchronous
- e.g. Rate Set at: 10Hz
- used by other activities

- Asynchronous

- <<active>>

- Asynchronous
- e.g. Rate Set at: 5 Hz

- <<active>>

- Asynchronous
- e.g. Rate Set at: 8 Hz

- EKF Pose Estimator

- Pose Estimator

- Path Information

- Path Planner

- K9 Rover

***Rover***

- Decision Layer

- Commanding and State Updates

- Obstacle Mapper

- ICRA2005 - I.A.N.

- April 18, 2005

## Navigation with Path Planning on Two Rovers 

- Navigation with Path Planning on Two Rovers

**Complex Algorithms on different Platforms**
**I/O, motion control**
**Trajectory Generation**
**Rough Terrain Locomotion**
**Odometry Pose Estimation**
**Stereo Processing**
**Visual Odometry**
**Navigation (Morphin)**
**Obstacle avoidance **
**Path Planning**

- ICRA2005 - I.A.N.

- April 18, 2005

## Designated Target Tracking for Single-Cycle Instrument Placement

- Designated Target Tracking for Single-Cycle Instrument Placement

**Integration of Complex Algorithms**
**I/O, motion control**
**Trajectory Generation**
**Rough Terrain Locomotion**
**Odometry Pose Estimation**
**Stereo Processing**
**Visual Odometry**
***Obstacle avoidance ***
**Mast Control**
**Visual Tracking**

- ICRA2005 - I.A.N.

- April 18, 2005

## And with a Simulated Rover

- ICRA2005 - I.A.N.

- April 18, 2005

## Acknowledgements

- CLARAty Team (multi-center)
_**Jet Propulsion Laboratory**_
-  ROAMS/Darts Team
-  CLEaR Team
-  Instrument Simulation Team
-  Machine Vision Group
-  Robotic Systems Group
_**Ames Research Center**_
-  K9 Team
_**Carnegie Mellon University**_
_**University of Minnesota**_

- ICRA2005 - I.A.N.

- April 18, 2005

## Thank you for your Attention

## Backup Slides

## Connecting Bodies and Joints

- ICRA2005 - I.A.N.

- April 18, 2005

## Some Results on Reusability

## Some Software Inter-operability Statistics 

| Algorithm Description | 1st Adaptation |  | Subsequent Adaptations (time in days) |  |  |
| --- | --- | --- | --- | --- | --- |
|  | On | Time | On | Adapt.Time | Tuning Time |
| 3D Extended Kalman Filter for Rover Pose Estimation | Rocky 8 | 60  | FIDO (JPL) | 1 | 3 |
| 3D Locomotion for wheeled vehicles | Rocky 7 | 30 | Rocky 8 (JPL) | 10 | 12 |
|  |  |  | FIDO (JPL) | 1 | 1 |
|  |  |  | K9 (ARC) | 4 | 1 |
| Morphin Navigator  | Rocky 8 | 75 | FIDO (JPL) | 1 | 1 |
|  |  |  | K9 (ARC) | 4 | 14 |
|  |  |  | ROAMS (JPL) | 1 | - |
| Mast Control Software | Dexter | 21 | Rocky 8 (JPL)  | 7 | 7 |
|  |  |  | FIDO (JPL) | 4 | 1 |

- ICRA2005 - I.A.N.

- April 18, 2005

## Code Reusability for Motion Control 

- Code Reusability for Motion Control

- ICRA2005 - I.A.N.

- April 18, 2005

## Code Resuability for Locomotion Example

- ICRA2005 - I.A.N.

- April 18, 2005

## Conclusions

- Use abstraction to master complexity
- Encapsulate and abstract hardware variations
- Provide multi-level access through Decision Layer for fault diagnosis and recovery
- Use domain expertise to guide design
- Make all assumptions explicit
- Stabilize external interfaces rapidly
- Document processes and products well
- Avoid over-generalization - define scope
- Encapsulate system specific runtime models
- Do not comprise performance - least common denominator solutions are unacceptable in hw/sw interactions
- Standardize Hardware

- ICRA2005 - I.A.N.

- April 18, 2005

## Examples of CLARAty Reusability

*Controlled_Motor_Impl*

- ControlledMotor

- Linear_Axis

- Joint

- Non-Resuable Layer

- R8_Motor

- Fido_Motor

- Sim_Motor

- Trajectory

- Trajectory_Generator

- Mz<Type>

- HCTL_Chip

- Widget_Board

- Widget_Motor

- PID Controller

- Counter

- DIO

- Analog Out

- Analog In

- MSI P460

- MSI P430

- MSI P415

- MSI P430

- ResuableHW reusable
- Non-reusable

- R7_Motor

- LM629_Chip

- R7_MC_Board

- ICRA2005 - I.A.N.

- April 18, 2005

## Adapting to a Rover

- Hardware Drivers

- Generic
- Functional Layer

- Rocky 8
- Specialized
- Classes
- &
- Objects

- Connector

- Decision Layer

- Multi-level access Connector

- Rocky 8
- Models/
- Heuristics

- ICRA2005 - I.A.N.

- April 18, 2005

## Supported Platforms

**Solaris**

**x86**

- Ames

- JPL

***Rocky 8***

**x86**

**VxWorks**

**Linux**

***Rocky 7***

***K9***

***ROAMS***

***FIDO***

- CMU

**Linux**

***ATRV***

**x86**

- JPL

***FIDO***

**VxWorks**

**x86**

- JPL

**ppc**

**VxWorks**

- JPL

**Linux**

- ICRA2005 - I.A.N.

- April 18, 2005

## CLARAty Team

**NASA Ames Research Center**
- Maria Bualat
- Sal Desiano
- Clay Kunz (*Data Structure Lead*)
- Eric Park
- Randy Sargent
- Anne Wright *(Cog-E & Core lead)*
**Carnegie Mellon University**
- David Apelfaum
- Reid Simmons *(Navigation lead)*
- Chris Urmson
- David Wettergreen
**University of Minnesota**
- Stergios Roumeliotis
- Yukikazu Hidaka

**Jet Propulsion Laboratory**
- Max Bajracharya (34) *(Cog-E &  Vision lead)*
- Edward Barlow (34)
- Antonio Diaz Calderon (34)
- Caroline Chouinard (36)
- Gene Chalfant (34)
- Tara Estlin (36) *(Deputy Manager & Decision Layer lead)*
- Erann Gat (36)
- Dan Gaines (36) *(Estimation Lead)*
- Mehran Gangianpour (34)
- Won Soo Kim (34) *(Motion lead)*
- Michael Mossey (31)
- Issa A.D. Nesnas (34) *(Task Manager)*
- Richard Petras (34) *(Adaptation lead)*
- Marsette Vona (34)
- Barry Werger (34)
**OphirTech**
- Hari Das Nayar

- ICRA2005 - I.A.N.

- April 18, 2005

## Summary

- CLARAty provides a repository of reusable software components at various abstraction levels
- It attempts at capturing well-known robot technologies in a basic framework for researchers
- It publishes the behavior and interfaces of its components
- It allows researchers to integrate novel technologies at different levels of the architecture
- It is a collaborative effort within the robotics community
- It will run on multiple heterogeneous robots

- ICRA2005 - I.A.N.

- April 18, 2005

## Component Analysis

- Objects

- Members

*Generic Physical Comp*

- Estimator

- State Machines

- Object
- Services

*Creates*

***Private***

***Public***

*Links to*

- State Handler

- Sub-object

*Internal *
*Implementation*

- Local Estimation

- Queries

- HW Object

- - optional link

- ICRA2005 - I.A.N.

- April 18, 2005

## Collaborations

**CLARAty**

- MIT
*Dubowsky*

- RMSA Competed Tasks

- CMU
*Stentz*

- U. Michigan
*Borenstein*

- JPL
*Matthies*

- ARC
*Roush*

- U. Washington
*Olson*

- Jet Propulsion Lab

- CMU

- NASA ARC

- U. Minnesota

- MDS
*Lai*

- Inst. Placement
*Kim*

- Long Traverse
*Huntsberger*

- Validation Tasks

- Auto. Science
*Gat*

- New NRA
- Competed Tasks

- WITS
*Backes*

- Rover Infrastructure
*Baumgartner*

- Science Sim
*M. Lee*

**MTP**

**CICT**

- ALERT
*Matthies*

- OASIS
*Castano, Judd*

- Adv. Avionics
*Bolotin*

**R&TD**

- Craters
*Cheng*

- MIT
*Williams*

- CMU
*Glymour/**Ramsey*

- JPL
*Nesnas*

- Ohio State U.
*Li*

- ARC
*Dearden*

- CLEaR
*Estlin/Fisher*

- Instr. Placement
*Pedersen*

- U. Survey
*Tunstel*

- Manipulation
*Backes*

- ROAMS
*Jain*

- ICRA2005 - I.A.N.

- April 18, 2005

- ICRA2005 - I.A.N.

- April 18, 2005

## Software Development Process

- Repository

*AFS Backbone*

- CMU

- JPL

- CLARAty

- Repository

- 3rd Party

- Releases

- Web

- UW

- ...

- VxWorks

- ARC

- Rocky 8

- FIDO

- Rocky 7

- Benchtops

- Benchtops

- Benchtops

- K9

- Repository

- ATRV

**Some CLARAty Statistics**
- ~170 Modules (reusable entity)
- ~31 Packages (module grps)
- ~3 rovers
- ~250,000 lines of  C++ code
- ~Java/scripts/ and models

*Authentication*

- K9

- ICRA2005 - I.A.N.

- April 18, 2005

## Example: Generic Controlled Motor

- Define generic capabilities independent of hardware
- Provide implementation for generic interfaces to the best capabilities of hardware
- Provide  software simulation where hardware support is lacking
- Adapt functionality and interface to particular hardware by specialization inheritance
- Motor Example: public interface command groups:
- Initialization and Setup
- Motion and Trajectory
- Queries
- Monitors & Diagnostics

*Time*

*Velocity*

- S- Profile

- Instantaneous Profile Change

- ICRA2005 - I.A.N.

- April 18, 2005

## Example: collaborative development for locomotor

- JPL - 1998

- CMU - 2002

- FutureARC - 2003

- Designed for Rocky 7
- Used Motor class
- Separated wheel control from locomotion
- Built-in pose estimation

- Generalized design for wheeled locomotors
- Full and partially steerable vehicle
- Used generic motor classes
- Implements fixed axle model
- Developed continuous driving
- Adapted to Rocky 8, Rocky 7,  and Sim

_**Version 1.0**_

_**Version 2.0**_

- Separated model from control
- Add separate locomotor state
- Add concept of wheel andsteerable wheel, Drive Cmd, Drive Sequence
- Adapt to ATRV, Sim,  Rocky 7, Rocky 8

_**Version 3.0**_

- Redesign/ mature

- Redesign/ mature

_**Version 4.0**_

- Use device and telemetryinfrastructureAdd adaptation to K9

- Add

- JPL - 2001

- ICRA2005 - I.A.N.

- April 18, 2005

## R8 Specific Rover Implementation

- Mast

- LeggedLoc

- Wheeled Locomotor

- Locomotor

- CoordMotionSystem

- Motor

- BBMotor

- ControlledMotor

- Analog_IO

- Digital_IO

- IO

- Manipulator

- Arm

- R8_Arm

- R8_Mast

- Implements general fwd &
- inv. kinematics & joint ctrl

- Specialized inv. Kinematics (overrides default)
- Attaches proper motors
- Attaches proper cameras for mast
- Adds filter wheel

- R8_Locomotor

- Attaches proper motors
- Restricts Steering to 2 wheels

- R8_Motor

- R8_Rover

- R8_Arm

- R8_Mast

- R8_Locomotor

***R8***

- Widget Board

- Widget AIO

- Widget DIO

- Widget Motor

- HCTL 1100 Chip

- Non reusable Code

- Reusable Code

- Trajectory

- Trajectory_Generator

- Timers

- ICRA2005 - I.A.N.

- April 18, 2005

## Capabilities of Wheel Locomotor

- Type of maneuvers:
- Straight line motions (fwd / bkwd)
- Crab maneuvers
- Arc maneuvers
- Arc crab maneuvers
- Rotate-in-place maneuvers (arc turn r=0)
- Driving Operation
- Non-blocking drive commands
- Multi-threaded access to the Wheel_Locomotor class – e.g. one task can use Wheel_Locomotor for driving while the other for position queries
- Querying capabilities during all modes of operation. Examples include position updates and state queries
- Built-in rudimentary pose estimation that assumes vehicle follows commanded motion

- ICRA2005 - I.A.N.

- April 18, 2005

## R7 Specific Rover Implementation

- Mast

- WheeledLoc

- LeggedLoc

- RBLoc

- Locomotor

- CoordMotionSystem

- Motor

- BBMotor

- ControlledMotor

- Analog_IO

- Digital_IO

- IO

- Manipulator

- Arm

- R7_Arm

- R7_Mast

- Implements general fwd &
- inv. kinematics & joint ctrl

- Specialized inv. Kinematics (overrides default)
- Attaches proper motors
- Attaches proper cameras for mast
- Adds filter wheel

- R7_Locomotor

- Attaches proper motors
- Restricts Steering to 2 wheels

- LM629Chip

- LM629Motor

- R7_Rover

- R7_Arm

- R7_Mast

- R7_Locomotor

***R7***

- Device Drivers

- VPAR10Board

- Non reusable Code

- Reusable Code

- ICRA2005 - I.A.N.

- April 18, 2005

## Why is robotic software “hard”?

- Software:
- Software is large and complex
- Has lots of diverse functionality
- Integrates many disciplines
- Requires real-time runtime performance
- Talks to hardware
- Hardware:
- Physical and mechanics are different
- Electrical hardware architecture changes
- Hardware component capabilities vary

- ICRA2005 - I.A.N.

- April 18, 2005

## What is CLARAty?

- CLARAty is a *unified *and *reusable ***software** that provides robotic functionality and simplifies the integration of new technologies on robotic platforms

*A research tool for technology development and maturation*

- ICRA2005 - I.A.N.

- April 18, 2005