# Presentation: 051338

## Xports: Network-Based Macro-Molecular Structure Determination

- R. Bramley, D. F. McMullen, J. Huffman
-  Indiana University - Bloomington
- I. Foster, G. von Laszewski, M. Westbrook
- Argonne National Laboratory
- E. Westbrook,
- Lawrence Berkeley Laboratory

## Overview 

- Application and goals
- X-ray crystallography; remote instrument control and collaboration
- Software architecture
- Explore use of DoE Common Component Architecture framework
- Extensions to non-compute components, network awareness
- Explore use of CoG Kits (commodity Grid components)
- Network requirements and research
- End-to-end reservation
- QoS requirements
- Capability for dynamic network access, control

## Application and Goals 

## Application and Goals 

- X-Ray crystallography to determine molecular structure
- Better resolution from high-energy, high-flux beam lines at LBL and ANL
- Current procedure:
- Ship crystal sample to LBL
- Lab tech mounts sample, runs predetermined scan procedure
- Data collected, written to DAT, shipped FedEx
- If sample is flawed/mounted incorrectly, multi-hour run wasted

## Application and Goals

- Desired procedure
- Scientist sends sample to LBL
- With scientist’s advice via teleconferencing, technician mounts sample
- Scientist sends initial set of scan commands
- Preliminary frames are filtered via local/remote components, get sent to scientist who manipulates images to check
*mounting is proper*
*sample is still good*
*whether sample is subject to twinning*
- Scientist sends new scan parameters - or terminates beam-line session
- At any time, scientist can allow remote colleagues to view partial results and confer

## Application

- Beam Source

- Sample Crystal

- Goniostat

- CCD Array

- Local
- Store

- CCD Data
- Collector

- User Ctl
- Scan Set

- Structure
- Analysis

- Local DSP,
- Buffering

- Collaborative
- Visualization

## Application

- Beam Source

- Sample Crystal

- Goniostat

- CCD Array

- Local
- Store

- CCD Data
- Collector

- User Ctl
- Scan Set

- Structure
- Analysis

- Local DSP,
- Buffering

- Collaborative
- Visualization

## Application

- Beam Source

- Sample Crystal

- Goniostat

- CCD Array

- Local
- Store

- CCD Data
- Collector

- User Ctl
- Scan Set

- Structure
- Analysis

- Local DSP,
- Buffering

- Collaborative
- Visualization

**Image Frames**
**to Struct. Analysis**

**Image Frames**
**from Local St**

**Image Frames**

**Ctl Signals**

**Status**

**User Ctl**

**Munged Frames**

## Xports Software Architecture 

- Common component architecture (CCA) mechanisms
- Utilize CoG Kits: Commodity Grid Kits
- CCA defines minimal feature set for HPC software “components” - but is not a component framework
- Provides for interoperability between components developed by different sites using different frameworks.
- CORBA, DCOM, Java Beans are related systems but don’t handle HPC and parallel components.  Fail on “minimum surface area” test.
- Basic ideas: CCAServices objects, and CCAPorts

## Component System Advantages

- Dynamic composition: run-time construction
- Run-time customization via configuration parameters
- Components may have state, multiple interfaces.
- Can move location without recompiling/relinking
- “Multicast” capability
- Supports composition of components in different languages and software systems

## CCA Ports

- Output ports provide methods to other components; example might be SendData( )
- Input ports are remote references to output port methods
***Only*** required method: PortInfo( ) giving port type and name as strings.  Type matching via the strings.
- Fast “direct connect” for components in same address space; has uses port = connected provides port

## CCA Services

- Each component is provided with “CCA.services” object
- Used by component to tell framework about the ports it has
- Access to services provided by framework for components
- Services methods include (SIDL interface descriptions)
- array<PortInfo> getProvidesPorts(  );
- array<PortInfo> getUsesPorts(  );
- Port getPort(in string name);
- registerUsesPort(in PortInfo name_and_type);
- addProvidesPort(in ProvidesPort inPort, in PortInfo name);
- releasePort(in string name);
- ComponentID getComponentID( );

**Notes:**
Provides Ports are provided by the component.  The component may implement the provides port itself or it may allocate an object that implements the provides port.  In either case, to tell the framework that a specific provides port is available, the component must call addProvidesPort( ) with the reference to the port instance and a PortInfo object naming and describing its type. 

Uses ports are named and specified by the component, but manufactured by the Framework. To tell the framwork that the component plans to use a port of a particular name and type it calls

## Indiana CCA Toolkit

- An implementation of CCA with
- Directory Service: for browsing remote directories containing data about components and their specifications.
- Registry Service: for browsing remote directories with data about running instances of components.
- Creation Service: instantiates another component.
- Connection Service: connects port of one component to another.
- Event Channel: publication and filtering of events. Push-based publisher channel/subscriber model with type filtering.
- Services above are themselves “pseudocomponents” - CCA components with some internal method access

## CCAT Extensions for XPorts

- CCA model based on computational components
- Components consisting of parallel programs
- Mixture of shared memory, distributed memory components
- Xports must extend to
- Remote instrument access
- Remote control
- Large-scale, multi-source data management
- Network capabilities access

## Xports Components

- Components match block model shown earlier
- Handle difficulty of proprietary HW/SW of beam-line equipment
- Control port takes user-specified scan parameters
- Frame output port multiplexed to avoid loss of data from network problems
- May require caching/sequencing component
- Data analysis components dynamically instantiated where most useful - on site, or at user location.
- User interface components: dynamically allow scripting, GUI, or auto-default as needed.

## Xports Components

- Network capabilities and CCA components
- Network aware computational/data store components
*Dynamic negotiation of protocols and capabilities, on a per-message basis if necessary*
*Ability to restore lost links, re-establish state*
*Interfaces to DiffServ, other systems (?)*
*Ability to vary QoS as analysis proceeds *
- Network status/statistics interface components
*Give user information about overall system*
- Co-allocation of multiple QoS streams with differing service levels (more later)

## Network requirements and research

- Bulk data transfer
- QoS requirements
- Network related component enhancements

## Bulk data transfer

- Rates: 12MB/sec for several hours
- Up to 16 MB/frame uncompressed (for 4k x 4k detector)
- 9-900 frames
- One frame per 1.5 sec
- Other apps (e.g., CMT) have one frame per 0.1 sec, but total bandwidth reservation time reduced by same factor
- Frames “demultiplexed” for reliability to
- local buffer/cache
- local or remote computational component
- mass storage component
-  (beam-line time rarer, more expensive than compute/network time).

## Xport QoS Requirements

- Multiple streams:
- CCD frame transfer: initial frames of critical importance
- 2 to 3 video cameras: operator, crystal, crystal alignment, user(s)
- Audio
- Real-time visualization and shared virtual spaces (collaborative CAVE data presentation and analysis)
- Co-allocation of multiple service levels
- Video and audio
- Bulk data transfer to/from HPSS and local cache
- Data transfer between components during execution of distributed computations

## Xports QoS Between Functional Blocks

## Network related component enhancements

- Determine and verify routing and link characteristics
- Components may connect temporarily, drop and reconnect
- During its lifetime a component should be aware of routing or link capacity changes
- Determine link characteristics (throughput, latency(?), optimum protocol and socket settings)
- DiffServ QoS support at component level
- Determine classes of service and permission to request them intra- and inter- domain (?!)
- Set a class of service for a flow
- Determine *ad hoc *performance of a “QoS” flow

## Xport Summary

- Application has full range of network requirements
- Good candidate for CCA component architecture extended to non-compute services
- Network requirements range from premium to default
- Require simultaneous management of multiple QoS streams with differing priorities.
- Component-to-component CCA port connection will have QoS requirements that vary dynamically