# Presentation: 131018

## QUALITY ASSURANCE TRAINING DOE O 414.1C AND10 CFR 830, SUBPART A

- Instructor:

## LEARNING OBJECTIVES

**Articulate the Underlying Principles of Quality Assurance**
**Discuss the interrelationships between Quality Assurance, Integrated Safety Management, and the Price-Anderson Amendments Act**
**List Basic Quality Assurance Requirements**
**Apply Quality Assurance to Your Work**

- 10/24/05   Revision 0

**Notes:**
At the end of this presentation, you should be able to:
 Articulate the Underlying Principles of Quality Assurance,
 Discuss the interrelationships between Quality Assurance, Integrated Safety Management, and the Price-Anderson Amendments Act. 
 List the Basic Quality Assurance Requirements, and
 Apply Quality Assurance to your specific work

## PRESENTATION OUTLINE

**Introduction**
- History
**Definitions**
**DOE Quality Assurance Program**
- Responsibilities
- Total Management System Overview
- DOE Quality Assurance Culture
- QA Rule and Order Integration
**Total Management System & Underlying Principles of:**
- Quality Assurance
- Integrated Safety Management
- Price-Anderson Amendments Act
**QA Program Basic Requirements & Criteria**
**Summary**

- 10/24/05   Revision 0

**Notes:**
 This presentation covers Quality Assurance and related requirements as addressed by DOE O 414.1C & 10 CFR 830, Subpart A.
 Start off with a short history of Quality Assurance, and 
 How the QA Rule and Order are integrated and where apply
 Definitions are important and we will review those that are necessary to understand the material presented.
 In the discussion of the DOE QA Program, we review the Functions and Responsibilities of an effective QA Program along with the DOE QA culture reflected in a Total Management System.
 This presentation will highlight the underlying principles of a Total Management System which is made up of: Quality Assurance, Integrated Safety Management, and the Price-Anderson Amendments Act and how they interrelate. 

## INTRODUCTION

**Brief History of Quality Assurance**
- 1954 AEC issued QC-1, Weapons Quality Policy
- 1969 AEC issued QAP standard for AEC reactors
- 1970 AEC published 18 QA criteria 10 CFR 50, App. B
- 1971 AEC & industry begin developing ANSI/ASME standards QAP requirements, guidance, & work practices for nuclear power plant design & construction
- 1972 ANS issued standard for admin controls during power plant operation
- 1975 ASME NQA committee established
- 1975 – 2004 Successive revisions to ASME QA requirements, ASME NQA-1-2004 most recent revision.
- 1980-2004 Successive revisions to DOE QA requirements, DOE O 414.1C (2004) most recent revision.
- 1994 DOE issued 10 CFR 830, Nuclear Safety Management

- 10/24/05   Revision 0

## INTRODUCTION (continued)

**The Quality Assurance Program is part of a Total Management System**
**Programmatic, but bottom line is it’s about**:
- People
- The work they perform, and
- Common sense

- 10/24/05   Revision 0

**Notes:**
 The principal measure of DOE’s performance is the quality of its products and services.  The Quality Assurance Program accomplishes this as part of a Total Management System.
 While the Total Management System encompasses the development, documentation, and maintenance of an effective Quality Assurance Program, the bottom line is it’s about People, the Work the perform, and Common Sense. 
 Each of you must review the expectations of your customer for each product or service and ensure your delivery meets them on the first try, safely, reliably, on schedule, and they meet or exceed the customer’s requirements, needs, and expectations. 
 Need to explain that the requirement for records exceeds common sense – but is required to demonstrate compliance with specified requirements.

## DEFINITIONS

## QUALITY DEFINITIONS

**Quality**:
- 	The condition achieved when an item, service or process meets or exceeds the users’ requirements or expectations.
**Quality Assurance (QA)**:
- 	All those actions that provide confidence that quality is achieved.
**Quality Assurance Program (QAP)**:
- 	A management system established to assign responsibilities and authorities, define policies and requirements, and provide for the performance and assessment of work.

- 10/24/05   Revision 0

**Notes:**
Before we begin, there are several terms that must be defined as they are similar and often confused or misused.  These are:
Quality is the condition achieved (or existing) when an item, service or process meets or exceeds the users’ requirements or expectations; 
Quality Assurance is all those planned and systematic actions that provide confidence that quality is achieved
Quality Assurance Program is the overall program or management system established:
 to assign responsibilities and authorities, 
define policies and requirements, and
 provide for the performance and assessment of work. 
The above definitions were taken from 10 CFR 830.

## QUALITY DEFINITIONS (continued)

**Graded Approach**:
- 		The process of ensuring that actions used to comply 	with a requirement are commensurate with:
- The relative importance to safety, safeguards, and security;
- The magnitude of any hazard involved;
- The life cycle stage, mission and characteristics of a facility;
- The relative importance of hazards; and
- Any other relevant factor.

- 10/24/05   Revision 0

**Notes:**
Continuing with definitions that were again taken from 10 CFR 830:
 You will see reference to the requirement to apply a graded approach to quality assurance in both the Rule and the Order.
 Graded approach means the process of ensuring that the level of analysis, documentation, and actions used to comply with a requirement in this part are commensurate with:
(1) The relative importance to safety, safeguards, and security;
(2) The magnitude of any hazard involved;
(3) The life cycle stage of a facility;
(4) The programmatic mission of a facility;
(5) The particular characteristics of a facility;
(6) The relative importance of radiological and non-radiological hazards; and
(7) Any other relevant factor.

## QUALITY DEFINITIONS (continued)

**System**:
- 	An integrated composite of people, products, and processes that provide a capability to satisfy a stated need or objective.
**Management System**:
- 	A set of interrelated processes that provides personnel what they need to effectively perform work which results in a product or service consistently meeting requirements and expectations.

- 10/24/05   Revision 0

**Notes:**
Continuing with definitions:
 A “system” is an integrated composite of people, products, and processes that provide a capability to satisfy a stated need or objective. There isn’t a definition for “system”  in either 10 CFR 830 or DOE O414.1C. This definition is taken from DoD 882E.
 A “Management System” is a set of interrelated processes that provide personnel what they need to effectively perform work which results in a product or service consistently meeting requirements and expectations.
 Performing work effectively includes meeting technical, safety, business, requirements and expectations.

## DOE QUALITY ASSURANCE PROGRAM

## RESPONSIBLITIES FOR QUALITY ASSURANCE

_**MANAGERS**_
**RESPONSIBLE AND ACCOUNTABLE FOR ALL ASPECTS OF**
** ****QUALITY OF PERFORMANCE, INCLUDING PLANNING, **
**ORGANIZATION, DIRECTION, CONTROL, AND SUPPORT**

_**ASSESSORS**__** **_
**RESPONSIBLE FOR EVALUATING EFFECTIVENESS**
**TO ACHIEVE QUALITY OF PERFORMANCE, **
**IDENTIFY DEFICIENCIES, AND **
**ENSURE CORRECTIONS**

_**PERFORMERS**_
**RESPONSIBLE FOR ACHIEVING QUALITY OF **
**PERFORMANCE SO AS TO ENSURE SAFETY AND RELIABILITY**

- 10/24/05   Revision 0

**Notes:**
 DOE has assigned roles to different levels of the organization to support quality work.  As you will hear stated many times, every individual in the organization is responsible for achieving quality in performing  his or her activities and is supported by management and assessment personnel.
 Specifically, senior management establish and cultivate principles that integrate quality requirements into daily work  while managers at all levels are responsible and accountable for quality of performance, including planning,  organization, direction, control, and support.  Management is also responsible for leadership and commitment to quality achievement and improvement within a framework of public, worker, and environmental safety.
 Again each worker or performer is responsible for achieving quality of performance so as to ensure safety and reliability on all tasks up and down the spectrum of work activities.
 A critical role is given to the assessors, who are responsible for evaluating effectiveness of quality efforts for performance,  identify deficiencies, and  ensure corrections are made.

## TOTAL MANAGEMENT SYSTEM EXPECTATION

**Total Management System** integrates systems, programs, and processes utilizing Rules, Orders,  other applicable Standards and Guides, and Best Industry Practices to achieve safe, reliable results.

- 10/24/05   Revision 0

**Notes:**
The Total Management System integrates systems, programs and processes utilizing Orders, Rules, other applicable Standards and Guides, and Best Industry Practices to achieve safe, reliable results.  

## TOTAL MANAGEMENT SYSTEM

-   **NQA-1****	****ISO**			**INPO       ****QC-1**
-   **Lessons Learned**		**ANSI       ASQ**
-   **Best  Practices ****	****	****Other Standards**

**TOTAL MANAGEMENT SYSTEM**

- 10/24/05   Revision 0

**Notes:**
This “alphabet soup” can be confusing but each is a building block to achieve the end result.
 DOE’s Total Management System is an “umbrella” program encompassing a number of DOE and industry standards and recommendations. These include:
  DOE Guides which provide supplemental information on Rules, Orders, notices, and regulatory standards;
  ISM, Integrated Safety Management and  ISMS, Integrated Safety Management System programs;
  PAAA, Price-Anderson Amendments Act requirements; 
  QC-1, DOE/NNSA Weapon Quality Policy;
  INPO, Institute of Nuclear Power Operations; 
  ISO, International Organization for Standardization; 
  ANSI, American National Standards Institute; 
  ASQ, American Society for Quality; and 
 ASME NQA-1, which will be discussed in more detail later along with other consensus standards.
 In particular, ASME NQA-1, Quality Assurance Program Requirements for Nuclear Facilities, is a  key national consensus standard for quality & safety with a unique focus on nuclear facilities & activities. Use of NQA-1 supports compliance with DOE QA Requirements and is the first choice for DOE nuclear facility applications.  NQA-1 establishes expectations for the performance of work and are reference points against which to measure excellence. 
 Other standards, where practicable and consistent with contractual or regulatory requirements, may be used as necessary to address unique/specific work activities (e.g., development and use of safety software or establishing the competence of a testing and calibration laboratory). 
 Standards include Federal, state, and local laws and regulations; Department Orders, Manuals, and Notices; and voluntary consensus standards that protect the environment and the safety and health of the workers and the public.  All such standards must be supportive of work, not barriers or extra burdens.

## QUALITY ASSURANCE PROGRAM IS PART OF A TOTAL MANAGEMENT SYSTEM

**Objectives/goals are to**:
- Ensure requirements & expectations are met
- Provide confidence in achieved results
- Comply with applicable regulations & requirements
**Fundamental to Total Management System**
**Applies to all work**

- 10/24/05   Revision 0

**Notes:**
 DOE’S Quality Assurance Program is part of a Total Management System which strives to:
 Ensure customer requirements & expectations are met
 Provide confidence in achieved results through planning, documentation and assessment (senior management, customers, regulators, public) and
 Ensure compliance with applicable regulations & requirements.  
 This is accomplished through:
Defining quality policies, purpose, objectives, requirements, standards, practices and procedures;
Scoping applicable items and activities;
Assigning  organizational responsibilities and authorities for  management, achievement and assurance of quality;
Designating  need for training and qualification of personnel; and
Providing for performance, assessment and continual improvement of work processes
 The Total Management System is intended to complement other DOE quality and safety programs to support and aid in the accomplishment of quality work.

## DOE QA CULTURE REFLECTED IN TOTAL MANAGEMENT SYSTEM

**Quality is everyone's responsibility **
**All contribute to quality in a concerted & cost effective manner**
**Achieving quality products & services is the measure of success**
**Quality achievement is verified or validated by either individuals or groups other than those who performed the work.**
**Quality is "built in" by providing every person:**
- Correct training, resources, and motivation
- Focus to safely do the job right the first time
**Assessors of work are:**
- Technically knowledgeable and performance oriented
- Focused on improving quality

- 10/24/05   Revision 0

**Notes:**
 DOE’s QA culture is reflected In It's Total Management System which expresses a fundamental expectation that all work will be performed safely and meets established requirements. Most importantly, quality is recognized as everyone's responsibility and all must contribute to quality in a concerted & cost effective manner.  Achieving quality products & services is the measure of success and quality is "built in" by providing every person correct training, resources, and motivation.  The focus is on doing the job right and safely, the first time.  And, assessors of work quality must be technically knowledgeable, performance oriented and focused on improving quality.
 The bottom line is that the Total Management System is not just a senior  management paper exercise.  It is intended for all who manage, perform and verify work. It consists of planning, documenting and implementing an integrated, cost effective and efficient quality management system to produce a quality product.  
 The System strives to achieve quality by these staff and workers possessing the competence, skills, knowledge, experience, training, resources and work ethic and motivation to do the job right.  It supports prompt detection, communication and correction of quality problems in within the Total Management System.
 This DOE QA culture is further enhanced by integrating Safety and Quality efforts which inherently complement each other. 

## The QA RULE, QA ORDER, & PAAA

- 10 CFR 830, Nuclear Safety Management
- Subpart A applies to all contractors at DOE nuclear facilities (including radiological facilities)
- DOE O 414.1 applies to DOE elements and contractors
- Price-Anderson Amendments Act establishes requirements, indemnification and enforcement

- 10/24/05   Revision 0

**Notes:**
 10 CFR 830, Nuclear Safety Management rule became effective January 10, 2001. It contains two subparts. Subpart A, Quality Assurance Requirements and Subpart B, Safety Basis Requirements.
 Safety basis means the documented safety analysis and hazard controls that provide reasonable assurance that a DOE nuclear facility can be operated safely in a manner that adequately protects workers, the public, and the environment.
 10 CFR 830, Subpart A applies to all DOE nuclear facilities (including radiological facilities).
 10 CFR 830, Subpart B applies to hazard category 1, 2, & 3 facilities and does not apply to < category 3 (radiological facilities).
 DOE O 414.1C applies to DOE elements and contractors and includes a requirement to integrate multiple QAP drivers imposed by QA regulations from other Federal agencies.
 Where a work activity, process or item is specifically identified as within the scope of 10 CFR 830, (or other regulations), that regulation prevails.
 In the event of a conflict between DOE O 414.1C and 10 CFR 830 (or other regulations), that regulation prevails.
 The Price-Anderson Amendments Act became effective in 1988. It establishes nuclear safety requirements, indemnifies DOE contractors, and provides DOE with enforcement authority.

## UNDERLYING PRINCIPLES

- QUALITY ASSURANCE
- INTEGRATED SAFETY MANAGEMENT
- PRICE-ANDERSON AMMENDEMENTS ACT

## QUALITY ASSURANCE (QA) UNDERLYING PRINCIPLES 

**Define policies and objectives. Ensure they are understood and accepted.**
**Specify roles, responsibilities, & authorities.  Ensure they are understood and accepted.**
**Specify and communicate expectations. Identify & allocate resources to achieve them.**
**Establish a culture that is conducive to quality achievement and continuous improvement.**

- 10/24/05   Revision 0

**Notes:**
 Turning first to the Department’s Quality Assurance (QA) Program, there are several underlying principles which are essential for every manager and worker to understand.  These include: 
 Defining, early on, the policies and objectives of QA for the task at hand.  It is also essential that these are understood and accepted by all workers and managers associated with the work.
 It is critical that QA Plans specify roles, responsibilities, & authorities so that all understand the “who” of responsibilities and, more importantly that, no omission in coverage has occurred. Again, it is essential to ensure they are understood and accepted.
 Up and down the line, the quality expectations for the work must be clearly specified, communicated, and understood. Setting high goals without the means to achieve them can easily happen.  Management must Identify & allocate sufficient numbers of qualified resources to achieve quality expectations.
 But all of this is a hollow paper plan unless each of you is committed to achieving these quality expectations and continuously strive to improve upon it. Emphasize on the continuous improvement aspects of total quality management is critical. Through the use of performance measurement and metrics you know if improvements are really being made.

## QUALITY ASSURANCE (QA) UNDERLYING PRINCIPLES (continued)

**Ensure people are competent at the work  they do.**
**Ensure the right people have the right tools and information at the right time.**
**Seek and use relevant experience.**
**Plan and control the work.**
**Use the right materials, tools, and processes.  Control any changes to them.**

- 10/24/05   Revision 0

**Notes:**
Continuing with principles also includes:
Ensuring workers who manage and execute work are qualified, trained and generally competent at the work they are asked to do;
Ensuring these qualified staff have the correct information to accomplish their assignment at the right time in the work cycle;
Seeking and using relevant experience and specialized expertise to help achieve quality expectations;
At all levels of the organization, plan and control the work and follow the plan in its execution;
Suspect/counterfeit items
Safety issue corrective actions
Software
Always, use the proper and correct materials, tools, and processes for every job and control and record any changes made to them.

## QUALITY ASSURANCE (QA)UNDERLYING PRINCIPLES (continued)

** ****A****ssess work to ensure it meets expectations.**
** ****Identify and remedy errors and deficiencies.**
** ****Periodically review management processes to improve effectiveness and efficiency.**

- 10/24/05   Revision 0

**Notes:**
And finally to:
Continually assess work to ensure it meets expectations;
Identify and remedy errors and deficiencies;
Periodically review management processes to improve effectiveness and efficiency; and
Control the introduction of suspect/counterfeit items in your work, track safety issue corrective actions, and control safety software issues. Remember software QA is a subset of QA.

## INTEGRATED SAFETY MANAGEMENT (ISM) - UNDERLYING PRINCIPLES

**Line management responsibility for safety**
**Clear roles and responsibilities**
**Competence commensurate with responsibilities**
**Balanced priorities**
**Identification of safety standards and requirements**
**Hazard controls tailored to work being performed**
**Operations Authorization**

- 10/24/05   Revision 0

**Notes:**
 The second key program that supports the Total Management System is Integrated Safety Management (ISM).  ISM systematically integrates safety into management and work practices at all levels so that work is accomplished while protecting the public, the worker, and the environment.  This is accomplished through effective integration of safety management into all facets of work planning and execution. In other words, the overall management of safety functions and activities becomes an integral part of accomplishing the work
 The Guiding Principles for Integrated Safety Management are to guide actions from development of safety directives to performance of work:
 Line Management is directly responsible for the protection of the public, the workers, and the environment. 
 Clear and unambiguous lines of authority and responsibility for ensuring safety are established and maintained at all organizational levels. 
 Personnel must possess the competence, experience, knowledge, skills, and abilities that are necessary to discharge their responsibilities. 
 Resources must be effectively allocated to address safety, programmatic, and operational considerations. Protecting the public, the workers, and the environment must be a priority whenever activities are planned and performed.
 Before work is performed, the associated hazards must be evaluated and an agreed-upon set of safety standards and requirements must be established which, if properly implemented, will provide adequate assurance that the public, the workers, and the environment are protected from adverse consequences.
 Administrative and engineering controls to prevent and mitigate hazards must be tailored to the work being performed and associated hazards. 
 Operations Authorization to define conditions and requirements to be satisfied before operations are conducted must be clearly established and agreed upon. 

## INTEGRATED SAFETY MANAGEMENT (ISM) – CORE FUNCTIONS

**Define the Scope of Work**
**Analyze the hazards**
**Develop and implement hazard controls**
**Perform work within controls**
**Provide feedback and continuous improvement**

- 10/24/05   Revision 0

**Notes:**
 The core functions of the Integrated Safety Management System that must be accomplished for all work include:
 Define scope of work which entails translating the statement of work into project tasks, prioritizing these tasks, and allocate qualified and adequate resources;
 Analyze and categorize hazards involves analyzing project tasks and identifying potential process and job hazards, categorizing the potential hazards, and reviewing hazard analyses; 
 Develop and implement hazard controls by identifying applicable laws, regulations, rules and standards and controls required to prevent/mitigate potential hazards.  These include the training and qualifications needed to perform the work,  engineering and administrative controls, and personal protective equipment. Work plan are developed that establish how the work will be accomplished, including the resources allocated to the tasks, a schedule of work activities, and a work breakdown structure.  These work plans must be and communicated to the workers; 
 Perform work within controls involves a prestart readiness assessment to verify that procedures, personnel, systems and equipment are ready to perform the project tasks. Authorization to start work and perform work within established controls again must be communicated up and down the line; and
 Provide feedback and continuous improvement by assessing the work being performed to identify issues and  identify and implement corrections for identified issues or concerns. 

## QA CRITERIA/ISM PRINCIPLES AND FUNCTION CROSSWALK TABLE

- 10/24/05   Revision 0

**Notes:**
 This table is taken from DOE G 414.1-2A and provides a cross walk between the combined the ISM Guiding Principles and Core Functions (DEAR clause) and the QA Criteria. The “X” indicates cross reference delineating (a) when the QA criteria and the principle/function have shared intent or (b) when the QA criterion is applied to the ISM principle or function.
 The complementary nature of Quality Assurance programs and Integrated Safety Management adds value to DOE efforts under the Total Management System.  Quality assurance and management systems provide processes and tools for ensuring that ISM objectives are achieved.  Both reflect a fundamental expectation that all work will be performed safely and meet established requirements. In this regard, the Total Management System ensures compliance with the approved safety standards set, so that the expectation for safe work within controls is met. This also ensures that workers, the environment, and the public are reasonably protected from harm.  
 At the working level, the DOE quality and safety requirements share a management systems approach to achieving their objectives. As such, the required system documentation for each ISM system description and QA Programs may be integrated into a single document to describe how the organization intends to implement the requirements.  Such supportive, complementary, and shared attributes of quality and safety management systems include:
 Senior ownership and line management responsibility with clear roles and responsibilities;
 Effective assessment methods;
 Balanced priorities (qualified resources & timing) and a graded approach to getting work done; and
 Competence, training, and qualification of management and execution staff, feedback and quality improvement.
 Again, DOE’s Total Management System ensures that all work will be performed safely and meet established requirements leading to quality results.

## PRICE-ANDERSON AMENDMENTS ACT (PAAA) - PURPOSE

**Promotes protection of the environment, health, and safety of the public and workers**
**Indemnifies (insures) DOE contractors against public liability due to a nuclear incident**
**Establishes nuclear safety requirements**
**Provides DOE enforcement authority**

- 10/24/05   Revision 0

## QUALITY ASSURANCE PROGRAM

- BASIC REQUIREMENTS
- AND
- CRITERIA

## BASIC QA PROGRAM REQUIREMENTS

- Rule & Order Requirements and Applicability
- Consensus Standards
- Integration of Requirements
- Responsibilities for a Documented & Approved Management System
- Flow-down of QA Program Requirements
- QA Criteria

- 10/24/05   Revision 0

## QA RULE & ORDER REQUIREMENTS AND APPLICABILITY

**Order**
- Enforceable by contract
- Applicability/Scope
- Responsibilities
- Definitions (consistent)
- Graded Approach
- Consensus Standards
- Integration of Requirements
- Flow-down of Requirements
- Criteria (same)
- Safety Software Requirements
- S/CI Prevention Requirements
- CAMP (Feds)

**Rule**
- Enforceable by law
- Applicability/Scope
- Responsibilities
- Definitions (consistent)
- Graded Approach
- Consensus Standards
- Integration of Requirements
- Flow-down of Requirements
- Criteria (same)

- 10/24/05   Revision 0

**Notes:**
 Enforceable: The QA Rule is enforceable by law and the QA Order is enforceable when it is written into the contract.
 Applicability: 10 CFR 830 Subpart A applies to DOE nuclear facility contractors and DOE O 414.1C applies to all DOE elements and contractors. 
Where a work activity, process or item is specifically identified as within the scope of 10 CFR 830, (or other regulations), that regulation prevails.
 In the event of a conflict between DOE O 414.1C and 10 CFR 830 (or other regulations), that regulation prevails.
Responsibilities: Under the Rule, responsibilities for Federal employees is limited. The Rule primarily applies to contractors. The QA Order has expanded responsibilities for Federal employees.
Definitions: The definitions in the Rule and Order are consistent. Most of the definitions in the Order were taken from the Rule and reference the Rule.
 Graded Approach: Both the Rule and the Order require the application of a graded approach in implementing requirements. The basis for using a graded approach must be documented.
 Consensus Standards: Both the Rule and the Order require use of consensus standards as part of the QA Program.
 Integration of Requirements: Both the Rule and the Order require the integration of QA requirements with the Safety Management System.
 Flow-down of Requirements: Both the Rule and the Order require the flow-down of requirements to contractors and suppliers.
 Criteria: The 10 criteria contained in the Rule and the Order are the same. However, the Order specifically identifies SQA, S/CI, and CAMP as additional areas to be addressed in the QA Program. 

## CONSENSUS STANDARDS 

**Consensus Standards must be used to develop and implement the QA Program**
**Appropriate standards include ASME NQA-1, ISO 9001, ANSI/ASQ Z1.13**
- Must be practicable and consistent with contractual or regulatory requirements
**ASME NQA-1, Quality Assurance Requirements for Nuclear Facility Applications **
- Key national consensus standard for nuclear quality & safety
- Unique focus on nuclear facilities & activities
- Single most rigorous and comprehensive standard
- First choice for DOE nuclear facility applications
- Supports compliance with DOE QA Requirements
- Establishes criteria for the performance of work

- 10/24/05   Revision 0

**Notes:**
 National or international consensus standards must be used where practicable and consistent with contractual or regulatory requirements (e.g., 10 CFR 830). These standards are sometimes referred to as “voluntary standards.” However, once a practicable standard is adopted through regulation, code, contract, QAP, or procedure, compliance with the standard is required and is not voluntary.
 All standards used must be identified in the QA Program. Appropriate standards identified in DOE O 414.1C include the following:
 ASME NQA-1-2000, Quality Assurance Requirements for Nuclear Facility Applications (for nuclear-related activities); 
 Key national consensus standard for nuclear quality & safety
 Unique focus on nuclear facilities & activities  
 Single most rigorous and comprehensive standard
 First choice for DOE nuclear facility applications
 Supports compliance with DOE QA Requirements
 Establishes criteria for the performance of work

## DOE QA CRITERIA/NQA-1-2004 REQUIREMENTS CROSSWALK

- 10/24/05   Revision 0

**Notes:**
This table was produced from NQA-1-2004, Table 400, Titled 10 CFR 830 Subpart a, Dated January 10, 2001, 830.122, Quality Assurance Criteria. This Table identifies each of the ten DOE QA Rule and Order criterion and how they addressed by NQA-1-2004.
Table 400 of NQA-1 2004 contains the following comments.
DOE Criterion 1:  A DOE Quality Assurance Program will need to describe the management process for providing resources.
DOE Criterion 3:  A DOE QA Program will need to extend the requirements of NQA-1 to all conditions adverse to quality, not just significant conditions adverse to quality.
DOE Criterion 3:  A DOE QA Program will need to address collection and review of information, beyond deficiencies, to identify items, services and processes needing improvements.
DOE Criterion 5:  A DOE QA Program will need to address “work” as broadly as the DOE criterion, since the requirements for “work” are derived from multiple sources in the DOE Rule and Order.
DOE Criterion 9:  While audits per Requirement 18 of NQA-1 provide an input to management assessments, a DOE QAP will need to align with the intent, focus and concepts described in DOE Guide 414.1-1A, Management Assessment and Independent Assessment Requirements of 10 CFR 830.120 and DOE O-414.1, Quality Assurance, in order to meet the DOE Criterion.
 DOE Criterion 10:  Assessment as a DOE activity for a DOE QAP  will need to align with the intent, focus and concepts described in DOE Guide 414.1-1A, Management Assessment and Independent Assessment Requirements of 10 CFR 830.120 and DOE O-414.1, Quality Assurance, in order to meet the DOE Criterion.

## CONSENSUS STANDARDS - Continued

**ISO 9001-2000, Quality Management System**
- Applies to all types of organizations that are involved in non-nuclear activities
- Based on 8 quality management principles
- ISO chose these principles because they can be used to improve organizational performance and achieve success
**ANSI/ASQ Z1.13, Quality Guide for Research, 1999**
- Applies to non-nuclear research activities
- This document can be used in the development of a quality system for basic and applied research.

- 10/24/05   Revision 0

**Notes:**
 ANSI/ISO/ASQ Q 9001-2000, Quality Management System: Requirements 
This standard applies to all types of organizations that are involved in non-nuclear activities. It doesn't matter the size of the organization or what they do.
 It can help both product and service oriented organizations achieve standards of quality that are recognized and respected throughout the world
 ISO 9001-2000 standards are based on eight quality management principles. 
 Focus on your customers
 Provide leadership
 Involve your people
 Use a process approach
 Take a systems approach
 Encourage continual improvement
 Get the facts before you decide
 Work with your suppliers
 ISO chose these principles because they can be used to improve organizational performance and achieve success.
 Organizations become “ISO 9001 certified” when they meet the Standards.
 In Part 4.3 of NQA-1-2004 there is a comparison between the ISO 9001 and NQA-1 requirements.
 ANSI/ASQ Z1.13, Quality Guidelines for Research, 1999 
This standard applies to non-nuclear research activities 
 This document can be used in the development of a quality system for basic and applied research. This includes fields like the biological, physical, and applied sciences use methods such as field investigation, laboratory experimentation, computer modeling, and theory formulation.
 Aligned with QA Rule and Order 10 criteria.

## CONSENSUS STANDARDS - Continued

**Standards for Specific Applications **
- Address unique/specific work activities such as:
- Development and use of safety software – ASME NQA-1-2000 (Required by DOE O 414.1C)
- Establishing the competence of a testing and calibration laboratory – ISO 17025

- 10/24/05   Revision 0

**Notes:**
 Standards for Specific Applications are used to address unique/specific work activities such as:
 Development and use of safety software where DOE O 414.1C invokes the use of ASME NQA-1-2000 or equivalent. 
 Establishing the competence of a testing and calibration laboratory may specify ISO 17025. 
 Establishing requirements for electrical equipment may specify IEEE standards.

## INTEGRATION OF REQUIREMENTS

- Quality Assurance requirements as defined in  Rule and/or QA Order must be integrated with Safety Management System including:
- Suspect/Counterfeit Items (S/CI)
- Corrective Action Management Program (CAMP)
- Safety Quality Assurance (SQA)

- 10/24/05   Revision 0

**Notes:**
 Where practicable and consistent with contract or regulatory requirements, quality assurance requirements must be integrated with other quality or management system requirements in DOE directives and external requirements and implemented as part of the QA Program.
 The QA Order specifically identifies the following be included in the QA Program:
 S/CI prevention process
 Safety Software Quality requirements
 Corrective Action Management Program 
 At the organizational or institutional level, the DOE quality and safety requirements share a management systems approach to achieving their objectives. As such, the required system documentation for each (ISM system description and QAP) may be integrated into a single document to describe how the organization intends to implement the requirements. In some cases, the local DOE office (Site Office or Field Element) and contractor may determine that it is expedient to maintain both the ISM system description and the QAP. In these cases, as a minimum, the implementing mechanisms that are described in each should be integrated to the maximum extent practical, and the system description and QAP should cross-reference these procedures as applicable. For example, the processes and procedures for conducting management assessments should be referenced in both the QAP and the ISM system description. Likewise, a single process (e.g., procedures and plans) that satisfies quality and safety requirements at the facility level and the activity level should be used. 
 Note: Refer back to the QA CRITERIA AND ISM REQUIREMENTS CROSSWALK slide.

## ROLES AND RESPONSIBILITIES

- EH roles and responsibilities for a documented and approved Management System:
- DOE’s independent element responsible for safety of the public, worker and environment
- Develops and maintains QA policy requirements, guides, and standards for all DOE work
- Provides advise and assistance to DOE elements and contractors implementing DOE QA policy
- Identifies and proposes resolutions for crosscutting QA issues
- Develops and maintains the EH QAP

- 10/24/05   Revision 0

**Notes:**
From the QA Order, EH has the following responsibilities.
These roles and responsibilities are supported by DOE’s Implementation Plans for both DNFSB Recommendations 2002-1 and 2004-1. 

 EH is DOE’s independent element responsible for safety of the public, worker and environment

 EH develops and maintains QA policy requirements, guides, and standards for all DOE work

 EH provides advise and assistance to DOE elements and contractors implementing DOE QA policy

EH identifies and proposes resolutions for crosscutting QA issues

 EH is also responsible for developing and maintaining the EH QAP

## ROLES AND RESPONSIBILITIES - Continued

- PSO roles and responsibilities for a documented and approved Management System:
- Ensure HQ, field elements, and contractors implement the requirements in the QA Order
- Set priorities and schedule for compliance
- Provide direction and resources, including prioritization, for implementing the QA requirements
- Review and approve QAPs or other documents that include implementation approaches for the QA requirements
- Design and implement line management oversight programs

- 10/24/05   Revision 0

**Notes:**
 From the QA Order, the PSOs have the following responsibilities.

 These roles and responsibilities are supported by DOE’s Implementation Plans for both DNFSB Recommendations 2002-1 and 2004-1. 

 Ensure HQ, field elements, and contractors implement the requirements in the QA Order
 Set priorities and schedule for compliance – 2002-1
 Provide direction and resources, including prioritization, for implementing the QA requirements
 Review and approve QAPs or other documents that include implementation approaches for the QA requirements – 2004-1
 DOE O 226.1, Implementation of DOE Oversight Policy assigns PSOs responsibility to design and implement line management oversight programs for DOE Headquarters and field organizations.

## FLOWDOWN OF QA PROGRAM REQUIREMENTS

- Quality Assurance Program must:
- Flow-down requirements to subcontractors and suppliers to ensure criteria are satisfied
- Flow-down should employ graded approach so that subcontractor requirements are commensurate with subcontractor scope of work
- Requirement in both the QA Rule and Order

- 10/24/05   Revision 0

**Notes:**
 The Quality Assurance Program must:
 Flow-down requirements to subcontractors and suppliers to ensure criteria are satisfied
 Flow-down should employ graded approach so that subcontractor requirements are commensurate with subcontractor scope of work
 Requirement in both the Rule and QA Order
 The QA Rule states that the QAP must: . . . Describe how the contractor responsible for the nuclear facility ensures that subcontractors and suppliers satisfy the QA criteria. If applicable, procurement documents and contracts for items and services provided to facilities covered by QA Rule should include a statement informing the supplier or subcontractor that it is subject to the QA Rule and of the potential for enforcement actions under 10 CFR 820 (PAAA). Suppliers and subcontractors are not required by the QA Rule to submit their QAPs to DOE for review and approval; rather, it is left to the contractor to determine the methods for ensuring that procured items and services meet requirements and perform as expected. 
 DOE G 414.1-2A, Attachment 1 provides a QA Assessment Plan Template that includes further guidance to ensure that QA requirements flowdown to subcontractors and suppliers.

## QUALITY ASSURANCE CRITERIA

_**BASIC**_
_**REQUIREMENTS**_

**PERFORMANCE**

**MANAGEMENT**

**WORK PROCESSES**
**DESIGN **
**PROCUREMENT **
**INSPECTION & ACCEPTANCE TESTING**

**MANAGEMENT ASSESSMENT**
**INDEPENDENT ASSESSMENT**

**PROGRAM**
**PERSONNEL TRAINING & QUALIFICATION**
**QUALITY IMPROVEMENT**
**DOCUMENTS & RECORDS**

**ASSESSMENT**

- More detail on each of the 10 criteria to follow.

- 10/24/05   Revision 0

**Notes:**
Focusing now on QA, there are ten criteria (same in the QA Rule & Order) that fall under three categories (Management, Performance, and Assessment). These are the basic Quality Assurance requirements or criteria.  
Program - Management
Personnel training & qualification - Management
Quality improvement - Management
Documents & records - Management
Work processes - Performance
Design - Performance
Procurement - Performance
Inspection & acceptance testing - Performance
Management assessment- Assessment
Independent assessment- Assessment
Additionally, with the increasing size, complexity, and information technology dependence within DOE, there are three additional areas that must be addressed and folded into the above criteria.  They are:
Suspect/Counterfeit Items (S/CIs) Control & Prevention (criteria #3, #7 & #8)
Safety Issue Corrective Actions, and (criteria #3 & #9)
Safety Software Assurance (criteria #5 & #6)
These criteria will be discussed on the following slides.  Each criteria will be listed as it is stated in DOE O 414.C  and then discussed further to clarify the underlying intent.

## QUALITY ASSURANCE CRITERIA

## QA CRITERIA 

_**Management/Criterion 1—Program**_
- Establish an organizational structure, functional responsibilities, levels of authority, and interfaces for those managing, performing, and assessing work.
- Establish management processes, including planning, scheduling, and providing resources for work.
**Key Elements**
- Senior management ownership
- Performance objectives established
- Responsibilities, authorities, and interfaces established
- Individuals empowered
- Application based on graded approach

- 10/24/05   Revision 0

**Notes:**
 The first criterion deals with the QA Program itself as listed in the first bullet on the slide.
 Senior management must embrace ownership and establish an organizational structure, functional responsibilities, levels of authority, and interfaces for those managing, performing, and assessing work as well as planning, scheduling, and providing resources for work.  
 Performance objectives must be established and plans put in place for its achievement at the quality levels specified, in a safe and timely manner.  Also QA audits, surveillances, and nonconformance reports follow-up must be incorporated.
 Responsibilities, authorities, and interfaces must be established so there are clear lines of authority with functional responsibilities.  The QA organization must also be independent of line management organizations.
 Individuals must also be empowered to identify non attainment of quality performance objectives and involved in the resolution of problems within their area of expertise. Also Corrective Action Management Process and current status of corrective actions in tracking systems must be in place for non conformance issues.
 The graded approach is used for determining what QA management requirements are implemented for various types of work. This entails grading the application of QA requirements for activities, identifying consequences, requirements, and depth/extent/rigor necessary in application of those requirements are in place.  Controls and verifications are applied to an activity consistent with their importance to safety, cost, schedule, and success of the mission. 

## QA CRITERIA (continued)

_**Management/Criterion 2—Personnel Training and Qualification**_
- Train and qualify personnel to be capable of performing assigned work.
- Provide continuing training to personnel to maintain job proficiency.
**Key Elements**
- Affects all personnel
- Stimulate professional development
- Designed to address specific needs
- Maintain proficiency and promote improvement
- Conduct on-going review of training effectiveness

- 10/24/05   Revision 0

**Notes:**
 The second criterion deals with personnel training and qualifications as listed in the first bullet on the slide.
 Management is responsible for committing resources to facilitate the training and qualification processes for personnel hired or transferred into positions meet the appropriate job requirements.  This ensures personnel are capable of performing their assigned work and that adequate resources and processes support the selection, training, and qualification of personnel.  Before personnel are allowed to work independently, management must ensure they have the necessary experience, knowledge, skills, and abilities, based on: previous experience, education, and training; performance demonstrations or tests; completion of training or qualification programs; and on-the-job training.
 Management must also stimulate professional development for certifications and other required special qualifications (e.g., auditors, subject matter experts, nondestructive examination personnel, welders, etc.) and have management processes to establish and maintain such proficiencies.
 Training must be designed to address specific needs that assist personnel in acquiring knowledge of the correct and current processes and methods to accomplish assigned tasks. Staff must understand the fundamentals of the work, the associated hazards, the context within which the work is performed, and the reasons for any special work requirements. Such training can be grouped into three general categories: project-/task-specific, site-/facility-specific, and institutional.  Their initial training should prepare them to perform the job and continuing on-the-training should maintain and promote improved job performance.
 Finally, each level of an organization must conduct on-going reviews of training effectiveness.  In addition to quantifying its training and qualification needs, it must reassess its requirements, interfaces, training methods, training responsibilities, and duties to ensure the goal of quality and experience are achieved and maintained. Every manager and worker must also promote improvement for their work or functions. 

## QA CRITERIA (continued)

_**Management/Criterion 3—Quality Improvement**_
- Establish & implement processes to detect and prevent quality problems.
- Identify, control, and correct items, services, and processes that do not meet established requirements.
- Identify the causes of problems, and include prevention of recurrence as a part of corrective action planning.
- Review item characteristics, process implementation, & other quality-related information to identify items, services, & processes needing improvement.
**Key Elements**
- Standards and measures for performance
- Problem identification, control, resolution, and follow-up
- Cause analysis
- Continuous improvement
- Prevent problems

- 10/24/05   Revision 0

**Notes:**
 The third criterion deals with quality improvement for all work being planned, performed, and measured.  The focus is on improving the quality of products, processes, and services by establishing priorities, promulgating policy, promoting cultural aspects, allocating resources, communicating lessons learned, and resolving significant management issues and problems that hinder the organization from achieving its objectives. It is critical that:
 Standards and measures for performance to identify, control, and correct items, services, and processes that do not meet established requirements are in place.  Common quality performance analysis systems include  six sigma, metrics and indicators, trending, etc.
 There must be a system for problem identification, control, resolution, and follow-up.  This involves a review of item characteristics, process implementation, and other quality-related information to identify items, services, and processes needing improvement.  Quality problems identified are tracked through corrective action, identification of cause, and the effective resolution of a quality problem to prevent its recurrence.
 Cause analysis Identifies the “why” of problems, and includes prevention of recurrence as a part of corrective action planning. Quality problems typically involve: 
 documenting dispositions for repairing, reworking, inspecting, or testing items; 
 replacing or returning items to suppliers, scrapping the items, or using them as is; 
 changing process parameters or procedures; 
 eliminating substandard conditions; and/or 
 changing the management system or methods for achieving compliance. 
 Continuous improvement assesses quality improvement process to detect and prevent quality problems.  These processes communicate lessons learned and performance information.  They must encourage employees to plan, develop, explore and implement new ideas for improving products, processes, and services. Improvement processes are most effective when each employee participates and should not be delegated to a particular person.
 Mangers must establish and implement processes to detect and prevent quality problems.

## QA CRITERIA (continued)

_**Management/Criterion 4—Documents and Records**_
- Prepare, review, approve, issue, use, and revise documents to prescribe processes, specify requirements, or establish design.
- Specify, prepare, review, approve, and maintain records.
**Key Elements**
- Scope of documents and records defined
- Documents identified and controlled
- Records identified and controlled

- 10/24/05   Revision 0

**Notes:**
The fourth criterion deals with documents and records as listed in the first bullet on the slide.
Documents and records are required to effectively manage, perform, and assess work. Management should identify any documents and records that must be developed and controlled and is responsible to provide the resources necessary to accomplish requirements. Specifics include:
The scope of documents and records must be defined.  Documents are required by organizations, projects, or programs to control policy and administrative and/or technical information. A document may describe work to be done; data to be used at different locations, by different people, in changing situations; and data to be controlled from time to time for reference purposes.
Documents must be identified and controlled.  A document control system should be established to supply the documents necessary for personnel to safely and correctly perform their assigned responsibilities. Document control systems ensure that the mechanisms developed to implement the safety management functions are properly prepared, controlled, and available for use and to control the preparation, review, approval, issue, control, and revision of documents.
Records must be identified and controlled.  Records contain information that is retained for its expected future value. Records should be sufficient to support technical and regulatory decisions and provide evidence that work was correctly performed. Records may be in a variety of forms (e.g., electronic, written, or printed; microfilm; photographs; radiographs; or optical disks). Typical records include procedures, plans, and manuals; training and qualification results; acceptance test results; technical/ regulatory correspondence; operational records; design basis descriptions, design review results, design revisions, and configuration management data; and quality problem resolutions. Records should be compiled and maintained in a records management system.

## QA CRITERIA (continued)

_**Performance/Criterion 5—Work Processes**_
- Perform work consistent with technical standards, administrative controls, and hazard controls adopted to meet regulatory or contract requirements using approved instructions, procedures, etc.
- Identify and control items to ensure their proper use.
- Maintain items to prevent their damage, loss, or deterioration.
- Calibrate and maintain equipment used for process monitoring or data collection.
**Key Elements**
- Management provides training, resources, and direction
- Employees are responsible for their work
- Review and improve processes

- 10/24/05   Revision 0

**Notes:**
 The fifth criterion deals with work processes which consist of a series of actions planned and carried out by qualified personnel using approved procedures, instructions, and equipment under administrative, technical, and environmental controls to achieve an end result.  These are:
 Managers are responsible for ensuring that staff have the training, skills (including knowledge and understanding of the capabilities of the processes being used), equipment, work process documents, and resources needed to accomplish their work.. Managers must ensure that the following are clearly identified and conveyed to workers before they begin work:  customer and data requirements for the work and final product; hazards associated with the work and safety, administrative, technical, environmental, and quality controls to be used; technical standards applicable to the work and final product;  acceptance criteria applicable to the work; and final product, and  procedures for verification of the completed work using established criteria.  Procedures and work instructions must be documented and controlled. The scope and detail of documentation is commensurate with the complexity and importance of the work, the skills required to perform the work, the hazards and risks or consequences of quality problems in the work and the need to meet regulatory and contract requirements. Control of processes, skills, hazards, and equipment should be clearly specified, understood, and fully documented which serves as the point of integration for Total Management System.
 Workers are responsible for the quality of their work and must do their work correctly the first time, in accordance with established procedures and work instructions. Because workers are the best resource for contributing ideas for improving work processes, products, and services, they should be involved in work process design, process evaluation (pre-job briefing), and providing the feedback necessary for improvement
 Implement methods adequate for ensuring that work is performed consistent with technical standards, administrative controls, and other hazard controls and that the core functions and guiding principles of ISM addressed in work processes to include: approved instructions, procedures, and other appropriate means for the work processes must meet regulatory or contract requirements;  identify and control items to ensure their proper use; maintain items to prevent their damage, loss, or deterioration; and  calibrate and maintain equipment used for process monitoring or data collection
 Work items are suspect when inspection or testing indicates that it may not conform to established Government or industry-accepted specifications or national consensus standards.  A process for the identification and control of items should be established and implemented to: prevent the use of incorrect or defective items, identify and control Suspect/Counterfeit Items (S/CIs), and provide for the control and maintenance of items.  Software computer programs, procedures, and associated documentation and data pertaining to the operation of a computer system must also be reviewed. For software test planning, personnel should identify the activities necessary to show the acceptability of the software against the approved requirements and to verify the functionality of the software.

## QA CRITERIA (continued)

_**Performance/Criterion 6—Design**_
- Design items and processes using sound engineering/scientific principles and appropriate standards.
- Incorporate applicable requirements and design bases in design work and design changes.
- Identify and control design interfaces.
- Verify/validate the adequacy of design products using individuals or groups other than those who performed the work.
- Verify/validate work before approval and implementation of the design.
**Key Elements**
- Design based on sound engineering/scientific principles
- Changes must be justified
- Interfaces must be identified and coordinated
- Verification must be conducted

- 10/24/05   Revision 0

**Notes:**
 The sixth criterion deals with design as listed in the first bullet on the slide.
 A design process should be established that provides appropriate control of design inputs, outputs, verification, configuration and design changes, and technical and administrative interfaces. 
 Designs must be based on sound engineering/scientific principles and appropriate standards, incorporating applicable requirements and design bases in design work and design changes.  The design of  safety class and safety significant (SSCs) items, software, and processes should be subject to design process controls and verification requirements appropriate to the level of risk the items present to the public, the worker, the environment, and project success. Designs should provide for appropriate acceptance, inspection, testing, and maintenance criteria to ensure continuing reliability and safety of the items. The designer should consider the expected use and life expectancy of the items to allow appropriate disassembly and disposal requirements to be addressed.  Design records should include documentation such as design inputs, calculations, and analyses; engineering reports; design outputs; design changes; design verification activities; and other documents that provide evidence that the design process was completed correctly.
 Changes must also be justified with configuration management controls in place for identifying and defining the configuration items in a system (i.e., software and hardware), controlling the release and change of these items throughout the system’s life cycle, and recording and reporting the status of configuration items and change requests.
 Design interfaces must also be identified and coordinated to control design. 
 Verification must be conducted before approval and implementation of the design.  There must be independent verification/validation of the adequacy of design products using individuals or groups other than those who performed the work. 

## QA CRITERIA (continued)

_**Performance/Criterion 7—Procurement**_
- Procure items and services that meet established requirements and perform as specified.
- Evaluate and select prospective suppliers on the basis of specified criteria.
- Establish and implement processes to ensure that approved suppliers continue to provide acceptable items and services.
**Key Elements**
- Procured items and services must perform as specified and meet requirements
- Use qualified suppliers
- Acceptance of procured items and services in accordance with specified methodology

- 10/24/05   Revision 0

**Notes:**
 The seventh criterion deals with the procurement process that ensures items and/or services provided by suppliers meet the requirements and expectations of the end user. The procurement process should be planned and controlled to ensure that the:
• end user’s requirements are accurately, completely, and clearly communicated to the supplier; 
• supplier, designer, and end-user requirements are met during the production phase; and 
• proper product is delivered on time and maintained until use. Procurement processes must prevent introduction of Suspect/Counterfeit Items (S/CIs) and provide a method to detect them before they are released for use
 Procured items and services must perform as specified and meet requirements.  The selection of procurement requirements should be commensurate with the importance of the end use of the purchased item or service.  All such items and services must meet established requirements and perform as specified.  A management process to identify how quality problems are identified, documented, reported, corrected, and prevented must be in place.
 The use qualified suppliers is imperative.  Managers must evaluate and select prospective suppliers on the basis of specified criteria and establish and implement processes to ensure that approved suppliers continue to provide acceptable items and services.
 Acceptance of procured items and services in accordance with specified methodology is of primary importance. Customer requirements/specifications must be determine and incorporated into the procured solicitation or delivery contract, with specific acceptance and rejection criteria.

## QA CRITERIA (continued)

_**Performance/Criterion 8—Inspection and Acceptance Testing**_
- Inspect and test specified items, services, and processes using established acceptance and performance criteria.
- Calibrate and maintain equipment used for inspections & tests.
**Key Elements**
- Inspection
- Conducted by, or for, the line organization
- Inspection criteria specified
- Acceptance Testing
- Conducted by or for the line organization
- Test and acceptance criteria specified

- 10/24/05   Revision 0

**Notes:**
 The eighth criterion deals with inspection & acceptance testing to verify that physical and functional aspects of items, services, and processes meet requirements and are fit for acceptance and use. 
 Performance expectations, inspections and tests are identified/considered early in the design process and/or specified in the design output and procurement documents. Before beginning work, personnel should check items to ensure they are correct and suitable for their intended applications. All staff should check their process output to verify that it meets or exceeds specified requirements.  
Inspections are conducted by, or for, the line organization with specific inspection criteria identified
Acceptance Testing is likewise accomplished by or for the line organization with test and acceptance criteria specified.
 All inspections/acceptance tests must be performed by technically qualified personnel who have the authority to access appropriate information and facilities to verify acceptance. These qualified personnel should be independent of the activities being inspected/tested and should have the freedom to report the results of the inspections/tests. Inspection/test results should be evaluated and verified by qualified personnel of the design organization to document that requirements have been satisfied.  All such inspections and acceptance tests results must be fully documented.

## QA CRITERIA (continued)

_**Assessment/Criterion 9—Management Assessment**_
- Ensure that managers assess their management processes and identify and correct problems that hinder the organization from achieving its objectives.
**Key Elements**
- Focuses on broad categories of management issues
- Addresses how well integrated management system is working
- Management responsibility and participation

- 10/24/05   Revision 0

**Notes:**
 The ninth criterion deals with management assessments as listed in the first bullet on the slide.
 Managers at every level must periodically assess their organizations and functions to determine how well they meet customer and performance expectations and mission objectives, identify strengths or improvement opportunities, and correct problems. Assessments should address the effective use of resources to achieve the organization’s goals and objectives. Management assessments should determine whether a Total Management System exists and whether it focuses on meeting both customer and performance requirements and strategic goals in.  
Focusing on broad categories of management issues;
Addressing how well integrated management system is working; and
Managements responsibility and participation.
 Managers are responsible for the conduct of management assessments and their direct participation by managers is essential to the success of the assessment process because they are in a position both to evaluate the organization as a total system and to effect change. 
 Delegating management assessment to a consultant or internal audit group is inconsistent with the requirement and will diminish its value to management. Personal involvement by management will yield the most meaningful information to be used in taking actions to maintain compliance and improve organizational performance 

## QA CRITERIA (continued)

_**Assessment/Criterion 10—Independent Assessment**_
- Plan and conduct independent assessments to measure item and service quality and the adequacy of work performance & to promote improvement.
- Establish sufficient authority and freedom from line management for independent assessment teams.
- Ensure that persons conducting independent assessments are technically qualified and knowledgeable in the areas to be assessed.
**Key Elements**
- Implemented by independent organization representing senior management
- Performance-based approach
- Technically knowledgeable personnel
- Focus on improving performance
- Scheduling based on performance

- 10/24/05   Revision 0

**Notes:**
 The tenth and final criterion deals with independent assessments of the organization’s programs, projects, contractors, and suppliers  to evaluate compliance performance of work processes. The results of independent assessments provide an objective form of feedback to senior management that is useful in confirming acceptable performance and should be used for identifying improvement opportunities.  The independent assessment process includes both compliance- and performance-based approaches that focus on results. Key elements include:.
 Independent assessments provide information on the quality of items, services, and processes produced by or for the organization. Consequently, independent assessment teams should report to a sufficiently high level in the overall organization. 
 Independent assessments to measure item and service quality and personnel performing independent assessments should have no direct responsibility for the work or organization they are assessing. 
 Independent assessment teams  must have sufficient authority and freedom from line management for independent assessment teams. Management must to ensure organizational independence from the work and access to levels of management authority capable of directing subordinate levels to take actions in response to the assessment results. 
 Personnel performing independent assessments should have the necessary technical knowledge to accurately observe and evaluate the activities and processes being assessed. 
 Assessments should focus on on improving performance as well as improving process effectiveness. Independent assessment personnel should base the evaluations on the approved system and not reinterpret or redefine the requirements. Performance-based approach for performance parameters should be used to support the improve quality and efficiency of operations and incorporated in the lessons learned program.
 Scheduling based on performance based on a risk model to determine assessment areas, schedule, and rigor.

## SUMMARYAPPLYING QA TO YOUR WORK

**Quality Assurance for all work is based on the development and maintenance of a single, integrated, effective Total Management System**
**Each manager and worker must review and comply with QA requirements in planning and executing work**
**Each employee is responsible for:**
- Achieving and maintaining quality
- Maximizing reliability and performance of work products, while
- Minimizing environmental, safety, and health risks and impacts

- 10/24/05   Revision 0

**Notes:**
 DOE strives for the highest quality in all its work based on the development and maintenance of a single, integrated, effective Total Management System for quality assurance.  Each manager and worker must embrace DOE programs for quality assurance and safety.  You must review and comply with quality assurance requirements in planning, procurements, and execution of work done by you or under your control.
 The bottom line is that each of us is responsible for achieving and maintaining quality, maximizing reliability and performance of work products, while minimizing environmental, safety, and health risks and impacts.

## DOE QA INFORMATION SOURCES

- _[DOE G 414.1-1A](http://www.directives.doe.gov/cgi-bin/explhcgi?qry1591734043;doe-132)_, 05/31/2001, Management Assessment and Independent Assessment Guide
**DOE G 414.1-2A**, 06/17/2005, Quality Assurance Management System Guide for Use with 10 CFR 830 Subpart A, Quality Assurance Requirements, and DOE O 414.1C, Quality Assurance
**DOE G 414.1-3**, 11/03/2004, Suspect/Counterfeit Items Guide for Use with 10 CFR 830 Subpart A, Quality Assurance Requirements
- _[DOE G 414.1-4](http://www.directives.doe.gov/cgi-bin/explhcgi?qry1591734043;doe-135)_, 06/17/2005, Safety Software Guide for Use with 10 CFR 830, Subpart A, Quality Assurance Requirements, and DOE O 414.1C, Quality Assurance

- 10/24/05   Revision 0

**Notes:**
There are several DOE Guides which can assist you in implementing DOE O 414.1C, Quality Assurance.  These are:
DOE G 414.1-1A, The revised Guide reflects current assessment practices, international standards, and changes in the Department of Energy expectations.
DOE G 414.1-2A, This Guide provides information on principles and practices used to establish and implement an effective quality assurance program or quality management system in accordance with the requirements of 10 CFR 830
DOE G 414.1-3, This Guide provides guidance to assist DOE/NNSA and its contractors in mitigating the safety threat of suspect/counterfeit items (S/CIs).
DOE G 414.1-4, This Guide provides acceptable methods for implementing the safety software quality assurance requirements of draft DOE O 414.1C, Quality Assurance

## BACKUP SLIDES

## SUPPLEMENTAL HIGH-RELIABILITY PRINCIPLES

- Highly-Reliable Operational Performance
- Individual Attitude and Responsibility
- Performance Assurance
- Organizational Performance Improvement

- 10/24/05   Revision 0

**Notes:**
The following was provided in the 8/5/05 memorandum from Bruce Carnes regarding reinvigorating ISM as part of the 2004-1 Implementation Plan.
Based on experience and learning over the past 10 years since the inception of Integrated Safety Management, DOE established the following four supplemental high-reliability principles to be used, along with the existing ISM guiding principles, to help develop the appropriate context or environment for effective implementation of ISM systems within the DOE and at its sites and facilities for 2005 and beyond:
A. Highly-Reliable Operational Performance. Organizations achieve sustained, high levels of operational performance, encompassing all DOE and contractor activities to meet mission, safety, productivity, quality, environmental, and other objectives. High-reliability is achieved through a focus on operations, quality decision-making, open communications, deference to expertise, and systematic approaches to eliminate or mitigate error-likely situations.
B. Individual Attitude and Responsibility. Every individual accepts responsibility for safe mission performance. Individuals demonstrate a questioning attitude by challenging assumptions, investigating anomalies, and considering potential adverse consequences of planned actions. All employees are mindful of work conditions that may impact safety, and assist each other in preventing unsafe acts or behaviors.
C. Performance Assurance. Competent, robust, periodic and independent oversight is an essential source of feedback that verifies expectations are being met and identifies opportunities for improvement. Performance assurance activities verify whether standards and requirements are being met. Performance assurance through conscious, directed, independent reviews at all levels brings fresh insights and observations to be considered for safety and performance improvement.
D. Organizational Performance Improvement. The organization demonstrates excellence in performance monitoring, problem analysis, solution planning, and solution implementation. The organization encourages openness and trust, and cultivates a continuous learning environment.