# Presentation: 391014

## Component Technology for Terascale Simulation Software

- PI: Rob Armstrong (SNL, _rob@ca.sandia.gov_)
- Technical Leads:
- Frameworks – Scott Kohn (LLNL)
- Components – Lois Curfman McInnes (ANL)
- Parallel Data Redistribution – Jim Kohl (ORNL)
- Applications Integration – David Bernholdt (ORNL)
- Presented by L.C. McInnes at the
- TOPS Center Kickoff Meeting
- August 20, 2001

**Component Technology for ****Terascale Simulation Software**

- Common Component Architecture
**Participants:**
- ANL
- Indiana Univ.
- LANL
- LLNL
- ORNL
- PNNL
- SNL
- Univ. of Utah
**Partners:**
- Climate Modeling
- Quantum Chemistry
- Combustion Modeling

## Bringing Components to High Performance Computing:Challenges and Goals

**Today: **
**Single-purpose, monolithic,**
**tightly-coupled**
**parallel codes**

**Currently DOE Apps Are:**
- One-off, stove-pipe, few participants
**But Also:**
- Scalable, and low latency
- Large & rich legacy investment

**Goals**
- Enable plug-and-play parallel simulations
- Establish component "marketplace" with applications partners
- Extend commodity component technology for HPC

**Challenges**
- Practical  parallel HPC component model
- Integrate legacy software investment
- Connection to Grid components
- Multi-language

**CCA**

***HPC Component***
***    ******Framework***

***Scientific***
***    ******Components***

***“******MxN” Parallel Data***
***    ******Redistribution***

***Applications***
***    ******Integration***

## Traditional Approach to Software Interoperability

- Direct interfacing between different packages/libraries/apps
- Public interfaces are unique
*Many-to-Many* couplings require *Many **2* interfaces
- Often a heroic effort to understand the details of both codes
- Not a scalable solution

- SUMAA3d

- DAs

- GRACE

- Overture

- Hypre

- Trilinos

- ISIS++

- PETSc

- Data / mesh software

- Linear solvers

## Common Interface Specification

- Reduces the *Many-to-Many* problem to a *Many-to-One* problem
- Allows interchangeability and experimentation
- Difficulties
- Interface agreement
- Functionality limitations
- Maintaining performance

- SUMAA3d

- DAs

- GRACE

- Overture

- Hypre

- Trilinos

- ISIS++

- PETSc

## Motivation

- Existing component architecture standards such as CORBA, Java Beans, and COM  do not provide support for parallel components.

- CORBA/Java

**CCA**

- MPI

- Latency
- between
- components

- Desire to build scientific applications by hooking together  components
- DOE Common Component Architecture (CCA) provides a mechanism for interoperability of high-performance components developed by many different groups in different languages or frameworks.

## Requirements for a High-Performance Component Architecture

- Simple/Flexible
- to adopt
- to understand
- to use
- Support a composition mechanism that does not impede high-performance component interactions
- Permit the SPMD paradigm in component form
- Meant to live with and rely on **other** commodity component frameworks to provide services ...
- e.g., JavaBeans, CCM, ...

## History and Participants

- CCA forum originated in 1998 within DOE’s DOE2K program; open to everyone interested in HPC components
- Active CCA forum participants *
- ANL - Lori Freitag, Paul Hovland, Kate Keahey, Lois McInnes, Boyana Norris
- Indiana Univ. - Dennis Gannon, Randall Bramley
- LANL - Craig Rasmussen
- LLNL - Scott Kohn, Gary Kumfert, Tom Epperly
- ORNL - David Bernholdt, Jim Kohl
- PNNL - Jarek Nieplocha
- SNL - Rob Armstrong, Ben Allan, Jaideep Ray
- Univ. of Utah - Steve Parker
- Terascale - Robert Clay, Lee Taylor
- CCTTSS team is a subset of CCA forum members

- * To join CCA forum mailing list, send "subscribe cca-forum" to _majordomo__@z.ca.__sandia__.__gov_

## CCA Approach

- CCA specification dictates a basic set of interfaces (and corresponding behaviors) that components should implement to be CCA compliant.
- Metaphor from visual programming (e.g., AVS); instead of data-flow, ports are linked with interfaces
- Provides/Uses design pattern
- Components are manipulatable in a framework.
- CCA specification doesn’t dictate frameworks or runtime environment
- Create components that are usable under a variety of frameworks
- Provide a means for discovering interfaces
- Specifically exclude *how* the components are linked; that is the job of a framework
- Provide language-independent means for creating components (via SIDL)

**Notes:**
The CCA bends over backwards not to legislate methodology either to the framework or component writers
while preserving the ability to do high-performance intercomponent links.

Iunknown provides for interface negotiation -- IDL will provide for reflection.

## Focus of CCTTSS

- HPC component specification and framework
- Unified reference framework implementation targeting both SPMD and distributed environments
- Tools for language interoperability via SIDL
- Suite of scientific components
- Linear and nonlinear algebra, optimization, mesh management, scientific data,visualization, steering, fault tolerance, scientific application domains, etc.
- Parallel data redistribution
- Model coupling, visualization
- Applications integration
- General outreach to the scientific community
- Close feedback loop for users/developers of CCA technology
- Collaborate with climate and chemistry applications domains as well as other ISIC centers and applications

## Ports preserve the high performance of direct connections plus the versatility of distributed object systems.

- Allow directly connected interfaces: the next component is a few function calls away (w/ SIDL).
- Adapters can create network-distributed objects out of the same components without altering them.

- Component1

- Component2

- Provides/
- Uses Port

- Component1

- Component2

- Uses Port

- ProvidesPort

- Network

## CCA Concept of SCMD (SPMD) Components

**A**

**A**

**A**

**A**

**B**

**B**

**B**

**B**

- MPI

- MPI

- Process

- MPI application using CCA for interaction between
- components A and B within the same address space

**Adaptive mesh**
** ****component**
**written by user1**

**Solver component**
**written by user2**

**Direct**
**Connection**
- supplied by
- framework at
- compile/runtime

- CCA Concept of SCMD (SPMD) Components

**Proc1**

**Proc2**

**Proc3**

**etc...**

## CCA Collective Port Modularizes Processor/Data Decomposition 

- parallel
- visualization component

**collective port**
**connecting M procs **
**with N procs**

- Combining previous parallel component with another
- parallel component in a different framework

- container composed of
- mesh and solver
- components

## It’s not that simple: Detailed standards are being developed

- Language interoperability
- Scientific IDL (SIDL) and Babel tool
- Scott Kohn (LLNL: skohn@llnl.gov)
- Distributed components
- Line protocols, inter-framework compatibility
- Dennis Gannon (IU: gannon@cs.indiana.edu ), Gary Kumfert (LLNL: kumfert@esaki.llnl.gov)
- Scientific data components
- Structured/unstructured mesh data
- Lori Freitag (ANL: freitag@mcs.anl.gov)
- Data redistribution
- Repartitioning, inter-parallel program communication, model coupling
- Jim Kohl (ORNL: kohlja@ornl.gov)

## Collaborations with Other SciDAC Groups

- CCA does not pretend to be experts in all numerical algorithms, just to provide a standard way to exchange component capabilities.
- Medium of exchange: interfaces
- Equation Solver Interface Forum (ESI) has made headway toward defining interfaces for linear algebra
- _http://z.ca.__sandia__.__gov__/__esi_
**Need experts in various areas to define sets of domain-specific abstract interfaces**
- nonlinear solvers, optimization, meshes, visualization, scientific application domains, etc.
- e.g., Newton-Krylov methods

- This
- means
- you!

## Major Activities in Early Years

- Year 1
- Assess requirements of other SciDAC groups; promote and support CCA approach through tutorials, workshops, etc.
- Define interfaces and develop various base component prototypes
- Complete candidate SCMD framework specification and implementation; complete candidate distributed computing CCA runtime environment
- Year 2:
- Deploy base component prototypes; integrate base component implementations; define second level component extensions; propagate language interoperability
- Define applications interfaces; begin production componentization efforts for chemistry and climate applications
- Year 3:
- Deploy integrated SCMD/distributed framework; deploy component repository
- Streamline component and framework implementations; implement fault tolerance and recovery mechanisms; generalize and standardize system specifications

## Simple CCA Demo of Optimization Solvers

- solution
- function
- gradient

**u**
**f**
**g **

- Compute **min f(u)**

-  CCAFFEINE – Common Component Architecture Fast Framework Example in Need of Everything
-  reference framework under development by B. Allan et al. (SNL)
-  http://www.cca-forum.org/cafe.html
-  TAO – Toolkit for Advanced Optimization
-  http://www.mcs.anl.gov/tao

**Optimization**
**via TAO solvers**

**Local Physics,**
**Discretization**

**g**

**f**

**u****i+1**

**u****o**

**CCAFFEINE Framework**

- component 1

- component 2

## CCA Compliance in TAO

- Paradigm shift; both TAO and the application become components
- Each is required to provide a default constructor and to implement the CCA component interface
- contains one method: “setServices” to register ports
- All interaction between components is through ports
- Application *provides* a “go” port and *uses* “taoSolver” and “MPI” ports
- TAO *provides* a “taoSolver” port
- There is no “main”  routine

## Another Demo (work in progress)

**Driver**

**Optimization**
**u****i+1**** = u****i**** + ********s ...**

**Mesh**

**Visualization**

**Local Physics,**
**Discretization**

**Runtime**
**Options**

**Linear Solver**
**H s = g**

**H**

**g**

**s**

**g**

**H**

**f**

**u****i+1**

**u****i+1**

**x,c**

**u****o**

**x,c,u****i+1**

**u****o**

**Data Redistribution**

**CCAFFEINE Framework**

## More Information