# Presentation: 486191

## SciDAC SSS Quarterly Report

**Sandia Labs**
**January 25, 2005**
**William McLendon**

- Sandia is a multiprogram laboratory operated by Sandia Corporation, a Lockheed Martin Company,for the United States Department of Energy’s National Nuclear Security Administration under contract DE-AC04-94AL85000.

## APItest - Overview

**Current Release : v0.2.5**
**Test Driver tool for testing software**
**Ordered tests**
**API Testing**
**Available for download:**
_**[ftp://ftp.sandia.gov/outgoing/apitest/](ftp://ftp.sandia.gov/outgoing/apitest/)**_

## Work Since Previous Meeting

**October : Development meeting at ORNL**
**Imported APItest into OSCAR**
**Started developing some APItests**
**Fixing bugs**
**Finishing up User Guide**

## APItest - Features

**Scripted Test Driver**
**XML based tests**
**Object Oriented**
**Easy to define new test types.**
**Conditional test ordering**
**Batch Scripts**
**Can call other batches**
**Built In Test Types**
**Shell**
**Script**
**SSS**
**Fully scripted in Python**
**Portable**

**Can set environment variables for tests.**
**SUID Test Capability**
**CAUTION!**
**Timeouts**
**User Interface Types**
**Web Browser**
**Command Line**
**Archived Results Browsing**
**Test Results Export**
**Save Test results to an XML file.**
**Open Source**
**LGPL**

**Notes:**
Feature list page.

## APItest - Requirements

**Packages required to run APItest:**
- Python 2.3 or greater
- lower versions can work but install is not as easy.
- ElementTree (_[www.](http://www.effbot.org/)__[effbot](http://www.effbot.org/)__[.org](http://www.effbot.org/)_)
- Twisted (_[www.](http://www.twistedmatrix.com/)__[twistedmatrix](http://www.twistedmatrix.com/)__[.com](http://www.twistedmatrix.com/)_)
- SSSlib (_[ftp://ftp.](ftp://ftp.mcs.anl.gov/pub/sss/)__[mcs](ftp://ftp.mcs.anl.gov/pub/sss/)__[.](ftp://ftp.mcs.anl.gov/pub/sss/)__[anl](ftp://ftp.mcs.anl.gov/pub/sss/)__[.](ftp://ftp.mcs.anl.gov/pub/sss/)__[gov/pub/sss/](ftp://ftp.mcs.anl.gov/pub/sss/)_)
- optional, for SSSlib tests.

**Notes:**
This is the list of packages required to run APItest.
The SSSLIB packages are not required if you don’t want to run any ssslib based apitests.

## APItest - Execution

**There are two ways to execute APItest**
**Command Line Only**
**$> apitest -f *****test_file.*****[*****apt|apb*****]**
**Web browser GUI  (Interactive)**
**$> apitest httpd**
**-i [--iroot] : root directory to look for tests**
**-p [--port] : port number (default = 2112)**
**-h [--host] : host (default = localhost)**
**Running with *****--help***** will display command line usage.**

**Notes:**
A few of the command line arguments.  There’s more... oh yes... many more.

## APItest - GUI (Top Page)

**Top level index page for APItest**
**APItest recurses from the [*****testroot*****] to find tests.**
**File Extensions:**
**TEST : .apt**
**BATCH : .apb**
**Batch tests shown in bold**
**Organized by directory**

## APItest - GUI (Test Run Execution)

**Quick status view of test execution.**
**This screen loads when the ****EXECUTE**** button is selected.**
**Color coded for easy viewing.**
**Currently running test is indicated.**
**Refreshes every 15 seconds until tests complete.**

## APItest - GUI (Test Run Summary)

**Summary of test results**
**Color coding**
**Batch Features**
**Cumulative summaries**
**Total # of tests**
**# that passed (green)**
**# that failed (red)**
**Hyperlinks to see more details about a test or batch.**

## APItest - GUI (Test Details - PASSED)

**Example of a test that was executed and passed.**
**Summary Data**
**Dependency Data**
**Only shown if test was run as part of a batch.**
**Output**
**Side-by-side comparison**
**Dependent on type of test.**

## APItest - GUI (Batch Summary)

**Clicking a hyperlink into a batch test gives us a summary view.**
**In this example, one test failed a dependency**
**FAILDEP**** in the status column indicates this.**
**Clicking on the scriptTest2.apt link will show us more data about this test.**

## APItest - GUI (Test Details - FAILDEP)

**Detailed view of scriptTest2.apt from previous slide.**
**Failed a dependency on cmdTest2.apt**
**Expected : ****FAIL**
**Actual : ****PASS**

## APItest - GUI (List of Test Runs)

**Clicking the ****View All Runs**** button gives this screen.**
**Test Runs are sorted by date and time.**
**Click links to view details of a particular run.**

## SSS - Service Directory Startup Test

**Notes:**
Example SSS service directory test sequence to check that /etc/init.d/rc.d/sss start|stop works and also that we can query the SD.

## APItest - Ongoing & TODO

**User Guide**
**Validation via file matching**
**Bug fixing**
**Configuration file**
**More SSS tests**
**Test Developer GUI**
**Remote test execution**
**Additional native test types**
**HTTP, TCP/IP, XMLRPC**
**Session management and security (Cookies?)**
**DB Connectivity**

## Slide 16

## APItest - Test Types

**CMD**
**Execute a command via the command line.**
**SCRIPT**
**Execute a script.**
**SSS**
**Execute a ssslib communication.**

## APItest - Example CMD Test

- <testDef>
**  **<info>
-     **Example test.  Runs ls -lr from /tmp, expects 0 exit status and any input.**
- </info>
-     <test type="**cmd**” wdir="**/tmp**">
-         <command interpreter="**/bin/ksh**">**ls**</command>
-         <input name="**argument**" format="**literal**">**-l**</input>
-         <input name="**argument**" format="**literal**">**-r**</input>
-         <input name="**stdin**" format="**literal**" />
-         <output name="**stdout**" format="**regexp**">**.***</output>
-         <output name="**stderr**" format="**literal**" />
-         <output name="**status**" format="**literal**">**0**</output>
-     </test>
- </testDef>

## APItest - Example Script Test

- <testDef>
-     <info>**Timeout test**</info>
-     <test name="**timestamp1**" type="**script**" timeout="**4**">
-         <command interpreter="**/bin/bash**" wdir="**/tmp**">
**for x in 1 2 3 4 5 6 7 8 9 10 **
**do **
**    ****echo "timeout_test :${x}: `date`”**
**    ****sleep 1**
**done**
-         </command>
-         <output name="**stdout**" format="**regexp**">**.*timeout.***</output>
-         <output name="**stderr**" format="**literal**" />
-         <output name="**status**" format="**literal**">**0**</output>
-     </test>
- </testDef>

## APItest - Example ENVVAR Test

- <testDef>
-     <info>
-     **This script executes but adds some stuff to the environment. **
**    ****NOTE: setting envvar OVERRIDES any inherited environment variables.**
**    **</info>
-     <test type="**script**">
-         <command interpreter="**/bin/csh**">**#!/bin/csh env**</command>
-         <input name="**envvar**" format="**literal**" key="**test_envvar_1**">**foobar**</input>
-         <input name="**envvar**" format="**literal**" key="**test_envvar_2**">**foo**</input>
-         <output name="**stdout**" format="**regexp**">
-             .*test_envvar_1=foobar\n.*test_envvar_2=foo\n.*</output>
-         <output name="**stderr**" format="**literal**" />
-         <output name="**status**" format="**literal**">**0**</output>
-     </test>
- </testDef>

## APItest - Example SUID Test

- <testDef>
-     <test type="**script**">
-         <command interpreter="**python**" uname="**wcmclen**" gid="**500**">
**#!/usr/bin/env python **
**import os **
**print os.getuid() **
**print os.getgid()**
**        **</command>
-         <input name="**argument**">**/tmp/testuser/**</input>
-         <input name="**argument**">**-l**</input>
-         <output name="**stdout**" format="**regexp**">**^27030$\n^500$**</output>
-         <output name="**stderr**" format="**literal**" />
-         <output name="**status**" format="**literal**">**0**</output>
-     </test>
- </testDef>

## APItest - Example SSS Test

- <testDef>
-     <info>**Sample sss query of the service directory**</info>
-     <test type="**sss**" destination="**service-directory**">
-         <input name="**sendbuf**">
**&lt;****get-location>****&lt;****location component='service-directory' host='*' port='*' **
**protocol='*' schema_version='*' tier='*'/>****&lt;****/get-location>**</input>
-         <output name="**recvbuf**" format="**regexp**">**.***</output>
-     </test>
- </testDef>

**Notes:**
NOTE: the &lt; tags are required here to replace the “<“ tags, since an XML parser will interpret the “<“ as the beginning of a tag.

Once I get some kind of XML editor into the APItest gui that can edit / create tests, I can automagically convert <‘s into &lt; values.