# Presentation: 101150

## Surviving an SQA Audit

**William E. Dyer, Jr.**
- Technologies Management Office Authoring Staff

## Preview of Coming Attractions

-  Review of “Text Book SQA”
-  “Experience is the best teacher”
- 		Some lessons learned from the 	American Time Use Survey 	(ATUS) CMM pilot project 	Software Quality Assurance Audit

## Text Book SQA

**SQA Expert Level**
**What is software quality assurance (SQA)?**
**Entry criteria/inputs**
**Exit criteria/outputs**
**Roles**
**References**
**Tasks**
**Measures**
**Very brief summary of SQA**

## What Is Software Quality Assurance (SQA)? 

**   **SQA provides visibility to management that the software products and processes in the project life cycle conform to the specified requirements and established plans.

** ****Entry Criteria/Inputs**
** ****Exit Criteria/Outputs**
** ****Roles**
** ****References**
** ****Tasks**
** ****Measures**

## Entry Criteria / Inputs

- Project requirements, standards,   organizational standards / processes, specifications
- Commitment to SQA Policy
- Project software quality goals
- Adequate resources committed to SQA

## Exit Criteria / Outputs

- Documented SQA Plans and procedures
- Trained SQA practitioners
- Results of  reviews, product evaluations, and     process audits
- Reports of problems or nonconformance on both products and processes
- Metrics of project and process status

## Roles

- Project Manager (PM) or other authority above software development organization: appoints and oversees SQA organization
- SQA Manager (SQAM), if appointed: leads SQA group
- SQA Group: team of individual SQA practitioners who implement this process
- Senior Management: periodically reviews SQA activities and resolves nonconformance issues when necessary

## References

- Census SQA Policy
- SQA Process, under SQA KPA
- SQA Plan Template, under SQA KPA
- IEEE/EIA 12207.0, Software Life Cycle Processes, Quality Assurance Process
- Capability Maturity Model for Software (SW-CMM)
- IEEE Std 1028, IEEE Standard for Software                            Reviews and Audits

## Tasks

- Establish SQA organization
- Select SQA tasks
- Create/Maintain SQA Plan
- Implement SQA Plan
- Create/Maintain SQA Procedures
- Identify SQA Training
- Identify/Select SQA Tools
- Improve Project SQA Processes

- Link to Task Appendix

## Measures

- Effort and funds expended for each activity
- Number of SQA reviews and audits conducted (planned vs. actual)
- Number of unresolved issues (elevated to PM) compared to all issues reported

## A very brief summary of SQA activities 

- Garner the authority for SQA
- Formulate a plan in accordance with corporate goals, CMM and IEEE standards and project software quality initiatives.
- Identify the role of an SQA auditor and the tasks to be fulfilled by the service.
- Report metrics

## Entry Level SQA

**As the software configuration manager for the American Time Use Survey, a CMM pilot project I was required to participate (fortunate to be involved) in the SQA audit process.**
**Following are some of the lessons learned from the experience.  **

## SQA - Another Great Adventure ...

-    Use the latest technology
-    Paper is unavoidable (Laura Croft finds a map)
-    A monster guards the prize (no teeth in SQA)
-    The entire crew shares the treasure as in:
- 	Pirates of the Caribbean,
- 	“Curse of the Black Pearl”
**Let’s talk process improvement ...**

**SQA - Another Great Adventure ...**

## Digging for Artifacts

- When I started gathering
- Documents for the ATUS
- SQA audit I did not know
- what an artifact was.  I
- learned quickly, though, most
- any schedule, memo, plan,

- document, program, change request, risk assessment, directory, software archive, paper copy, electronic media, database or other communication related to the project may be reviewed.
- My advice is to use SQA and audits as a tool to improve your process ***not*** because you are required to for a CMM KPA.

## All the Latest Technology

- The Census Bureau process improvement initiative is supported enterprise wide.
- We have the latest technology for promoting SQA including but not limited to:

-  Network resources (Lotus Notes +)
-  Project Management Repository (PMR)
-  Corporate Metadata Repository (CMR)
-  Survey Documentation tools (SUDDS)
-  Hardware, software and training

## Tomb Raiders Have Maps!

- The SQA Audit for ATUS was quite a learning experience in ***“what not to do.”***
- As one of the pilot projects for CMM at the Census Bureau We were thrust into the SQA arena with little planning or forethought.  We did well considering

- many of our documents and procedures were developed
- from scratch specifically for the CMM pilot.
- Had I known then what I know now, we would have drawn
- a map at the beginning of the project.  So we would know
- what to expect when it came time for the SQA Audit.

## Preferred SQA Scenario

- Limit the scope of the SQA Audit.
- Have SDP and other process documents in place before beginning the project.
- Do not make promises you can not keep.
- Keep organized records and plan ahead for the SQA audit.
- Learn what the task really involves before you start.

## A Monster Guards the Prize ...			(yet SQA has no teeth) 

- What’s this?  A binary large object?
- A monster with no teeth?  A growing and unstoppable bureaucracy? New fad? Senior Management? Something to be feared or not?

- An SQA audit is not like dealing with the IRS; there is no penalty for being late, for making mistakes or poor performance.  SQA is an opportunity for growth and a chance to apply process improvement to your projects.

## Typical Change Control Form

- Change control forms are used to initiate requests for modifications of all sorts.
- Changes to specifications, corrections to the instrument or even to resolve discrepancies in plans vs. implementation uncovered during an SQA Audit.

**Typical Change Control Form**

**Notes:**
This request was made to change the name of the survey from TUS to ATUS the “American Time Use Survey.”

## The Crew Shares the Treasure

- SQA is like the movie:
-      **Pirates of the Caribbean, “Curse of the Black Pearl”**
- We all share in the benefits of process improvement, through performing activities like Software Quality Assurance Audits, or suffer the curse together.

## Process Task 1

- The PM appoints an individual or group responsible for SQA.
- SQA must have organizational freedom, authority, and independence of the software development activities to permit objective reporting.

**Establish an SQA organization**

- Return to Task

## Process Task 2a

**Select SQA tasks**
- The SQA Group selects the tasks that will be performed, such as those listed below:
- Reviewing software products, tools, and facilities against requirements and guidelines.
- Auditing software processes for compliance with standards, and with KPAs of the SW-CMM.  Example process evaluation and KPA verification tasks are part of the SQA process.
- Participating in Peer Reviews and Project Reviews (technical and management reviews) by providing status on compliance, problem areas, and risks.

- Return to Task

## Process Task 2b

**Select SQA tasks (cont.)**
- The SQA Group selects the tasks that will be performed, such as those listed below:
- Suggesting methods, standards, guidelines, and tools to be defined for the project and seeing that they are documented in the SDP.
- Reporting results of product evaluations and process audits to the PM, senior management, affected development groups, and the Software Engineering Process Group (SEPG) as appropriate.
- Collecting and reporting metrics on the status of cost and schedule, product evaluations, project quality, and audits.

- Return to Task

## Process Task 3a

**Create/Maintain SQA Plan**
- The SQA Group documents SQA plans in the Software Development Plan or in an SQA Plan.
- Consider:
- ·      Quality objectives, in measurable terms
- ·      Types of test and verification and validation (V&V) activities
- ·      Entry and exit criteria for development phases
- ·      Responsibilities of the SQA group
- ·      Resource requirements for the SQA group
- ·      Schedule and funding of SQA activities                (Task 3b)

- Return to Task

## Process Task 3b

**Create/Maintain SQA Plan (continued)**
-       SQA participation in project plans, standards, and procedures
-       Evaluations to be performed by SQA
-       Audits and reviews to be conducted by SQA
-       Standards and procedures used for SQA
-       Documenting and tracking noncompliance issues, and the        escalation procedure
-       Documentation that SQA is to produce
-       Method, audience, and frequency of providing feedback on SQA        activities

- Return to Task

## Process Task 4

**Implement the SQA Plan**
- The SQA Group performs the SQA function as defined in the SQA Plan.
- Problems or nonconformance with requirements are documented and reported to the PM or appropriate authority.
- Senior management addresses noncompliance issues that cannot be resolved within the project.

- Return to Task

## Process Task 5

**Create/Maintain SQA procedures**
- 	The SQA Group documents and maintains the procedures that describe the details of how SQA is performed.

- Return to Task

## Process Task 6

**Identify SQA Training**
- The SQA Group identifies training required to perform SQA tasks, based on project requirements.
- Training includes training of the SQA Group and SQA orientation for the software project team members.

- Return to Task

## Process Task 7

**Identify/Select SQA Tools**
- 	The SQA Group identifies tools required to perform SQA tasks, based on project requirements.

- Return to Task

## Process Task 8

**Improve Project SQA Processes**
- The SQA Group reviews SQA processes and identifies improvements and efficiencies for future use.
- SQA activities are reviewed with the PM and with senior management on a periodic and event-driven basis.
- Submit suggestions for improvement of this process to SEPG using Document Change Requests.

- Return to Task