# Presentation: 289428

## Trilinos SQA Training

- Robert Heaphy
- October 16, 2003

## ASCI Program Quality Policy

- ASCI Program Plan,  DOE/DP-99-000010592
- DOE Notice 203.1, Software Quality Assurance
- DOE Order 414.1A
**Quality Criteria, QC-1,  NNSA/AL**
- ASCI Software Quality Engineering: Goals, Principles, and Guidelines DOE/DP/ACI-SQE-2000-FDRFT-VERS2  (commonly called the Trilab GP&G)
- Sandia National Laboratories ASCI Applications Software Quality Engineering Practices (A version has appeared for each assessment year)

## Sandia Corporate Quality Policy

- DOE Order 414.1A
- 10 CRF 820
**Quality Criteria, QC-1,  NNSA/AL**
- (Sandia) Corporate Quality Assurance Program  CPR001.3.2
- (Sandia) Corporate Software Quality Assurance  CPR001.3.6
- QC-2  (which specifically allows the use of QC-1 instead)

## Sandia Quality Home Page

- Pubic Law 104-113    (Federal agencies should now use non government standards)
- DOE order accepting the use of non DOE standards
- ISO 9000   (Standard of choice)

## Other Quality Standards at Sandia

- CMM   (DOD) replacing MilSpec 2167A
-      + TSP, PSP, P-CMM...
- CMMI (to replace obsolete CMM)
- Baldrige

## Last Audit Results

- Final report being written 3 weeks ago and is not yet available.
- Trilinos and Zoltan did well!
- Primary problem was missing processes (such as training)
- Next audit may have changes, but no indications of future directions yet
- Management is concerned about next ASCI external audit (FY 2005?)

## Trilinos SQE Program

- Assumption:  There is a true and significant component of research in Trilinos.
- Assumption: Professionally, you already know how to perform your job well and are doing it.
- You don’t need 400+ practices in CMM or 618+ pages of CMMI!
- You need to get credit for what you already do.
- This is a generic quality program based on QC-1.
- GOAL: To continue a successful quality program which is minimally intrusive to developers and provides tangible benefits the development team.
- KEY:  Document our existing work practices using lightweight “process checklists”, then follow the checklists.

## Changes for this next audit cycle

- Improve, complete and **use** “process checklists”.
- Implement checklist tracking (team leaders).
- Suggest the use of **laboratory notebooks** & project notebooks.
- Write new version of Trilinos (& Zoltan) SQE/SQA documentation (Heaphy).
- Each package team will perform a Trilinos self assessment.
- Future changes (TBD):
-  traceability from requirements to maintenance  (QFD, perhaps).
-  continuous process improvement (PDCA/PDSA).

## Suggestion

- Avoid negative vocabulary in emails, documents, CVS comments, etc.
- Specifically, think how ASCI management (who may not understand programmer jargon) might react to terms like: hack, trick, and work around.

## Process Checklists

- Include necessary boilerplate from Trilinos SQA examples
- Divide each checklist into 5-10 key activities.  Provide a simple explanation of the activity and what is to be accomplished.
- Incorporate creating and saving appropriate artifacts
- Incorporate Peer Reviews and double checks
- Incorporate standard tools (Bugzilla, Doxygen, Bonzai, Purify, etc)
- Incorporate a few, simple metrics as appropriate
- Incorporate error proofing (future)
- Maintain master checklist under CVS control
- Maintain specific instantiation under CVS control
- Don't commit yourself unnecessarily! (Conduct code review vs conduct Fagan Review.  Is a team code review invalid if a single team member is missing? Do you really want require a "Coding Style Guide"?)

## Process Checklists - 2

- Follow the checklist every time.  If not
- Change the checklist through a formal process.  Do this to add improvements,  remove ambiguity, modify processes, etc.
- Obtain a formal waiver (please don't really do this!)
- Document procedure changes in lab notebook for R&D activities per QC-1! (A changed procedure must be signed and dated)
- Otherwise, failing to follow the checklist is a significant problem and a potential audit finding.
- NEW Track using spreadsheet/database (saved in CSV (ASCII) text for CVS)

## New Training Checklist -1 

- New Developers (within 1 month of start)
- Read "An Overview of Trilinos"
- Read "Trilinos Users Guide"
- Read "Trilinos Developers Guide"
- Read "Trilinos Developers Guide, Part 2, ASCI SQA"
- Read Package Developers kit
- Mentor Assignment with initial 2 hours mentoring (provide dates/times with mentor name):

## New Training Checklist -2

- All Developers (three year continuous training cycle)
- Over each three year cycle, each developer must receive and document the indicated training hours.  One hour credit is given for an hour of formal class attendance, conference attendance, team/project training, mentoring (either as mentor or student).  One hour credit is given for participation in the monthly Trilinos phone conferences. Three hours credit will be given for a published paper (author or coauthor), conference presentation, or for each hour of instructing a relevant training course.  Each developer will list below at least the required hours of training and maintain supporting documentation.
- Math and Algorithms training (60 hours)
- SQE, SQA training  (15 hours)
- Development tools training (5 hours)