# Presentation: 906684

## SSS Testing and Validation

**SSS Testing and Validation**

**February 23, 2003**
- Chicago, IL
**Neil Pundit**
**Erik DeBenedictis**
**William McLendon III**

- Sandia is a multiprogram laboratory operated by Sandia Corporation, a Lockheed Martin Company,for the United States Department of Energy under contract DE-AC04-94AL85000.

## Strategies for RT System Testing and Validation

QMTest

Application Test Packages

APITEST

Cluster Integration Toolkit

**Strategies for RT System Testing and Validation**
**QMTest**
**Application Test Packages**
**APITEST**
**Cluster Integration Toolkit**

**Overview**

## Distributed Runtime System Testing

**Complicated interactions**
**Approach to testing**
**Component Testing**
**“****Light” Applications**
**Performance / Functionality**
**Operational Profile**
**Stress Testing**
**Users expect a high-degree of quality in today’s high end systems!  Extensive (and smart) testing is critical to achieving this goal.**

## Review

**Review of topics covered at previous F2F meetings.**

## QM Test Review

**Scriptable test driver (Python >= 2.1)**
**Test Metadata defined with XML**
**HTTP interface (Zope)**
**HTML are customizable**
**Open source**
**Active development community**
**URL:**
**http://www.codesourcery.com/**

## QM Test Screenshot

## Application Test Framework Review

**Presented by Mike Carifio at November meeting**
**Test Framework + Packages**
**Package driver: GNU Make**
**Easy to use interface**
**make targets!**
**Archival of results**
**Used on production CPlant****TM**** systems**

## Application Test Packages

**Templates for Packages available**
**Easy to add new packages**
**Some Packages Already Included:**
**Linpack**
**NAS-Parallel Benchmarks**
**BLACS**
**MPI-Test**

## APITEST

## APITEST - Overview

**Unit-testing tool for network components**
**Idea from October 2002 meeting**
**Scriptable Tests (XML Schema Grammar)**
**Enabling Technology**
**Black/White Box Testing**
**Multi-Protocol Support**
**TCP/IP, SSSLib, Portals**

## APITEST Feature list

**Scriptable Tests**
**XML Schema**
**Test Entity Types:**
**Command Shell**
**Network Transmission**
**Test Dependencies**
**“****Parent”**
**“****Sibling”**

**Distributed Test Execution**
**Test Repetition**
**Delayed loops**
**Multiple Protocols**
**TCP/IP (SSSLib)**
**Portals**
**Result Handling**
**Historical Data**
**Archival**

## APITEST Concept

## APITEST Enables

**Flexibility in unit testing without a lot of coding**
**Black Box testing**
**Does a component support the API?**
**Fault injection?**
**RT Integration testing**
**“****Component” abstraction**
**Path analysis**
**White Box testing**
**Coverage Testing**
**Internal States of Component X**
**Unreachable States?**

## APITEST Example Script

## APITEST Summary

**New effort**
**Concept from October 2001 F2F discussions.**
**Good progress so far**
**Developed set of features and capabilities**
**Developed XML Schema definition for test entities**
**Implementation of v0.1a under way**
**C++**
**Xerces for XML parsing and Schema validation**
**Data structures 75% implemented at this time.**

## Cluster Integration Toolkit (CIT)

## Cluster Integration Toolkit

**Used for many integration and management tasks on CplantTM**
**Numerous cluster installs**
**Demonstrated scalability**
**Largest 1861 nodes.**
**daemon interfaces adaptable to XML over IP**
**command_daemon **
**status_daemon **
**James H. Laros III (**_**[jhlaros](mailto:jhlaros@sandia.gov)**__**[@](mailto:jhlaros@sandia.gov)**__**[sandia](mailto:jhlaros@sandia.gov)**__**[.gov](mailto:jhlaros@sandia.gov)**_**)**

**Notes:**
"An Extensible, Portable, Scalable Cluster Management Software Architecture”
Laros, Ward, Dauchy et. al. published in the proceedings of the International 
Conference on Cluster Computing 2002.

## Example status_daemon Config File

- <?xml version="1.0" ?>
- <config>
**  **<port>**2009**</port>
**  **<pid_file>**/tmp/csdaemon.pid**</pid_file>
**  **<log_level>**4**</log_level>
**  **<sd_local_checks>**on**</sd_local_checks>
**  **<sd_check_delay>**30**</sd_check_delay>
**  **<sd_report_delay>**300**</sd_report_delay>
**  **<sd_conn_timeout>**10**</sd_conn_timeout>
**  **<sd_parent_alert_retries>**3**</sd_parent_alert_retries>
**  **<sd_admin_log_dir>**/tmp**</sd_admin_log_dir>
**  **<sd_trending>**off**</sd_trending>
**  **<background>**0**</background>
- </config>

## status_daemon Example : Load Avg.

## What CIToolkit offer to SciDAC

**How much work to integrate an interface to allow these daemons to speak “SciDAC”?**
**Good integration test... **
**SSS XML Interface**
**Will be a good exercise**

## Conclusions...

**Testing and Validation**
**Framework for testing Distributed RT system.**
**Existing capabilities**
**Application Testing**
**Stress Tests**
**New capabilities...**
**APITEST**
**Quite a lot of work still to do...**
**CIToolkit integration with SciDAC?**