![](media/image1.png){width="2.0833333333333335in"
height="3.1666666666666665in"}

**[Table of Contents]{.underline}**

**Section 1: Audit Timeline and Auditor Responsibilities**

**Section 2: Using the Audit Guide**

**Section 3: Preparing Evaluation Sheets**

**Section 4: Evaluation Sheet Routing and Approval Process**

**Section 1: Audit Timeline and Auditor Responsibilities**

![](media/image2.png){width="6.364583333333333in"
height="2.6979166666666665in"}

1\. Prior to the audit, auditors should read through the contents of the
audit notebook to familiarize themselves with the audit process, recent
audit history of the facility, and the audit guide.

2\. An inbrief will be held Monday afternoon to introduce team members
and Center personnel and discuss the purpose of the audit. The time and
place of this meeting is usually communicated to team members via letter
or email. The schedule for the rest of the week will be determined
during the inbrief. All auditors are expected to attend the inbrief.

3\. Auditing will be conducted Tuesday through Thursday. Auditors must
cover the areas in the audit guide as a minimum, unless otherwise
directed by the Team Leader, or if the auditor determines that some
attributes of the guide are not applicable to the Center. The audit
guide represents the [minimum audit points]{.underline} (MAP). This
means that, at a minimum, all these items must be looked at. This is the
sample that has been selected to verify compliance with Agency
requirements and enable the Audit Team to evaluate the health of the
process, system, program, or organization. Auditors are required to
[submit their marked up audit guides]{.underline} to typing by 3 pm on
Thursday. See Section 2, Use of the Audit Guide.

4\. Audit team "tag-up" meetings, with Center personnel, will be held
every afternoon Tuesday through Thursday at a time determined prior to
the audit or during the inbrief. Auditors should be prepared to present
a brief review of the day's findings. Use prepared notes if necessary
(marked up audit guide). Similar meetings may be held in the morning at
the Team Leader's discretion.

5\. As findings are identified, evaluation sheets should be written and
submitted to typing as soon as possible. One of the benefits of this is
that it allows the entire team to see trends developing or to notice
related issues. Also, the draft report and outbrief are being worked
real time as findings are processed. In all cases, Evaluation Sheets are
to be completed before the end of each day. See Section 3, Evaluation
Sheet Preparation, and Section 4, Evaluation Sheet Routing and Approval
Process. Auditors should ensure to advise their Center counterparts/POCs
of their findings on the day that they are noted. Ensure that all
parties have a clear understanding of the issues.

6\. Auditing shall cease at noon on Thursday, and all audit findings
should be submitted to typing by 3pm. Unless specifically approved by
the Team Leader, Auditors will not start looking at new areas or records
after noon on Thursday. Auditors shall inform the Team Leader as soon as
possible if circumstances dictate that this schedule cannot be met.

7\. Auditors are required to [submit their marked up audit
guides]{.underline} to typing by 3pm on Thursday.

8\. Auditors in each audit area shall prepare a summary of the most
significant findings identified during the audit in their area and
submit it to typing by 4pm. (One summary for every audit area.) This
summary should be in bulletized form, enabling easy insertion into the
outbrief presentation prepared by the Team Leader.

9\. Auditors are encouraged to fill out the Auditor Quality Feedback
Form in the back of their audit notebook and submit them to typing or
the Team Leader. Providing this feedback is instrumental in improving
the audit process.

10\. The draft report is completed Thursday afternoon/evening. Auditors
may depart the Center when all of their findings have been through the
Evaluation Sheet Review and Approval Process and signed by all parties.

11\. An outbrief is held Friday morning at a time to be determined. All
auditors are expected to attend unless absence is approved by the Team
Leader. No actions are required by auditors during this meeting.

**Section 2: Using the Audit Guide**

1\. The audit guide is a tool that helps ensure that the proper
requirements are audited, and provides a means of recording the level of
compliance with each requirement. Auditors should not simply ask the
questions to Center personnel and record their answers regarding
compliance/ noncompliance. The questions are really for the auditor to
answer. It is the job of the auditor to gather enough evidence to be
able to determine whether compliance with each requirement is
satisfactory or unsatisfactory. Attributes and sub-attributes are
designed to help the auditor ask the questions that will gather this
information. Assessment of compliance should be based on observance of
objective quality evidence (documentation) when possible. Auditors
should examine written procedures, records, reports, databases, etc.
that demonstrate compliance. Firsthand observation of work in progress
is also an excellent way of verifying compliance.

2\. Audit guides shall be annotated to reflect which attributes were
reviewed and the status of compliance (satisfactory or unsatisfactory).
Evaluation Sheets should be filled out for instances where compliance is
not satisfactory, or where processes could be significantly improved
(see Section 3, Preparing Evaluation Sheets). If any attributes were not
reviewed, they should be clearly noted in the guide with reasons
provided. If the auditor has suggestions for improving the audit guide,
those comments are welcomed as well. Marked-up audit guides are to be
turned in at the conclusion of the audit.

**Section 3: Preparing Evaluation Sheets**

1\. When a finding is identified, the auditor shall fill out the blank
evaluation sheet (provided in the audit notebook). It is important to
fill out the Evaluation Sheets as findings are identified rather than
waiting until the end of the audit. Evaluation Sheets are to be turned
in at the end of each day. This will allow the entire team to see the
issues and note any trends or related issues. Each section should be
filled out in the following manner:

a\. [ITEM]{.underline}: Write the appropriate abbreviation for the audit
area (see b. below) followed by an incremental number. For example,
"LDE-1" would indicate the finding is in the audit area of Lifting
Devices and Equipment and is the first finding identified in that area.

b\. [AUDIT AREA]{.underline}: Write the title of the Audit Area that the
finding falls into. The Audit Areas are:

(1) FACILITY SAFETY (FAC)

(2) FIRE PROTECTION (FIRE)

(3) LIFTING DEVICES AND EQUIPMENT (LDE)

(4) PRESSURE VESSELS AND SYSTEM (PV)

(5) GIDEP (GID)

(6) MISHAPS (MIS)

(7) EMERGENCY PREPAREDNESS PROGRAM (EPP)

(8) EXPLOSIVES SAFETY (EXP)

(9) RANGE SAFETY (RS)

(10) UNDERWATER FACILITY/OPERATIONS (UW)

(11) SAFETY & HEALTH (SH)

(12) MANAGEMENT (MGT)

(13) SMA AUDITS AND REVIEWS (AR)

(14) ELECTRICAL SAFETY (ELEC)

(15) SPECIAL INTEREST ITEM (SII)

c\. [REFERENCES]{.underline}: Write the document number(s), title(s),
and issue date or version of the NASA document governing the audit area.
These may include Center documents as well.

d\. [REVIEWER (S)]{.underline}: Write the full first and last name of
each auditor involved in writing the finding.

e\. [PERSONNEL INTERVIEWED]{.underline}: Write the full first and last
name of Center personnel consulted/interviewed during investigation of
the finding. Write the person's title also.

f\. [AREAS TOURED]{.underline}: List the areas/facilities/buildings
toured.

g\. [OBJECTIVE QUALITY EVIDENCE (OQE) REVIEWED]{.underline}: List the
documented evidence observed that verifies compliance. This is normally
records, reports, procedures, or first-hand observation. Be sure to
include specifics such as the document number(s), title(s), and issue
date(s) of documents. Tours and discussions are not considered OQE;
these should be listed in the AREAS TOURED and DISCUSSION sections,
respectively.

h\. [CATEGORY]{.underline}: Check CRITICAL NONCOMPLIANCE, NONCOMPLIANCE,
OBSERVATION, or COMMENDATION/BEST PRACTICE in accordance with the
following guidance:

\(1\) CRITICAL NONCOMPLIANCE (CNC): Failure to follow requirements that
could lead to loss of life, serious injury to personnel, or damage to
high-value equipment; failure to follow a regulatory requirement (such
as OSHA); a broad-based failure to comply with requirements; or several
minor noncompliances in a given process, procedure, or practice.

\(2\) NONCOMPLIANCE (NC): A failure to comply with Federal, State,
local, Agency, or Center requirements that does not present the
potential for loss of life, serious injury, or damage to high-value
equipment; failure to follow requirements on a limited scale such that,
it does not indicate a systemic problem.

\(3\) OBSERVATION (OB): A condition that is not contrary to documented
requirements, but, in the judgment of the auditor, warrants improvement
or clarification. If the finding is not a significant concern or major
improvement, do not write a finding.

\(4\) COMMENDATION/BEST PRACTICE (C): A process or method that would
provide significant benefit to other Centers if adopted/implemented, or
acknowledgement of a requirement/process that the Center accomplishes
extraordinarily well. These must be something well beyond just meeting
requirements or doing their job.

i\. FINDING: Write a clear, concise sentence of the deficiency or
condition observed.

j\. [DISCUSSION]{.underline}: For both types of noncompliances, the
Discussion should clearly state the requirement that is violated. Cite
the reference with the paragraph containing the requirement, and include
any [supporting factual information about the specific situation at the
Center that will ensure that the Center understands the
violation]{.underline}. For Observations, the Discussion should identify
the weakness in the process and identify the benefits of implementing
the recommendation. For Noncompliances and Observations, it is also
acceptable to provide a recommendation within the Discussion on actions
to take to resolve the deficiency. For Commendations/Best Practices, the
Discussion should identify the aspects of the process or method that are
[exceptional]{.underline}.

2\. Write clearly and concisely. Each evaluation sheet should be limited
to one page. Define acronyms the first time used in the evaluation
sheet. Refer to example evaluation sheet provided in the audit notebook
for additional guidance.

3\. Written evaluation sheets should be reviewed by the auditor prior to
submittal to typing using the following checklist:

-   Is all necessary information present?

```{=html}
<!-- -->
```
-   Are the names and titles of Center personnel provided in the
    PERSONNEL INTERVIEWED section?

-   Is the appropriate FINDING CATEGORY checked?

-   Do all documents listed in the REFERENCES and OBJECTIVE QUALITY
    EVIDENCE REVIEWED sections contain the document number, title, and
    issue date?

-   Are items listed under OBJECTIVE QUALITY EVIDENCE REVIEWED actually
    OQE?

-   Is the FINDING one sentence that clearly describes the situation?

```{=html}
<!-- -->
```
-   Does the DISCUSSION:

```{=html}
<!-- -->
```
-   Relate to the FINDING and clearly explain the situation?

-   Contain detailed factual information and [avoid subjective
    statements]{.underline}?

-   Cite the requirement from the relevant reference that was violated
    (including paragraph number)?

-   Identify the benefits of implementing the recommendation? (for
    OBSERVATIONS)

-   Identify the exceptional aspects of the process/method (for
    COMMENDATIONS/BEST PRACTICES)

```{=html}
<!-- -->
```
-   Proofread to verify that:

```{=html}
<!-- -->
```
-   There are no spelling/punctuation errors, etc.

-   All acronyms are defined the first time they are used.

-   Names of people are not used in the Discussion.

-   The paragraphs in the DISCUSSION are not numbered.

-   Findings that are rolled-over from a previous audit or internal
    audit are annotated in the bottom of the DISCUSSION section.

    4\. The following page contains additional specific guidance for
    writing audit findings.

**INSTITUTIONAL/FACILITY/OPERATIONAL (IFO) SAFETY AUDIT**

**EVALUATION SHEET**

**GODDARD SPACE FLIGHT CENTER**

**OCTOBER 11-14, 2005**

**ITEM:** **AUDIT AREA:**

**REFERENCES**: *Include revision information and dates of references.*

**REVIEWER(S):**

**PERSONNEL INTERVIEWED:** *Names and titles*

**AREAS TOURED:** *Be specific*

**OBJECTIVE QUALITY EVIDENCE (*RECORDS AND DOCUMENTS*) REVIEWED:**

*Discussions and tours are not OQE.*

**CATEGORY:** *These must be consistent with the definitions and
consistently used all throughout the report.*

CRITICAL NONCOMPLIANCE NONCOMPLIANCE

OBSERVATION COMMENDATION/BEST PRACTICE

**FINDING:** *One sentence statement that clearly describe the
situation.*

**DISCUSSION:** *Provide justification based on facts for the finding in
this section. Do not make sweeping statements, nor speculate, without
facts to support them.*

*Provide perspective in order to evaluate significance of the issues on
an individual and consolidated basis. For example, "two danger tags not
filled out correctly", does not provide perspective. "Two incorrect tags
out of 40 reviewed" provides perspective. Quantify where possible.*

*Must have facts to show why it is a problem or why it is exceptional.*

*Keep it focused -- stick to topic.*

*Overall, the Evaluation Sheet should be able to pass the "outside
reader" test. It should stand on its own so that someone not at the
audit can still clearly understand the issues.*

**Section 4: Evaluation Sheet Routing and Approval Process**

![](media/image3.png){width="3.21875in" height="1.84375in"}

1\. This section details the process that each evaluation sheet normally
goes through from the time the auditor fills out the evaluation sheet to
its capture in the draft audit report. This process ensures that:

a\. Audit findings are correctly categorized and clearly presented.

b\. The content of each audit finding is consistent with the auditor's
intent.

c\. The Team Leader understands and agrees with the validity of each
finding.

A structured process utilizing a series of baskets and signatures is
used to obtain objective quality evidence to ensure that the above
statement is true.

2\. Three in-boxes/baskets/bins shall be set up in a convenient place in
the audit room and labeled "Typing," "Auditor," and "Team Leader."

3\. When an auditor observes a finding (Critical Noncompliance,
Noncompliance, Observation, or Commendation/Best Practice), the blank
evaluation sheet provided in the audit notebook shall be filled out (see
Section 3, Evaluation Sheet Preparation).

4\. Completed evaluation sheets shall be placed in the bin labeled
"Typing." The audit support team will then record the finding (in
paraphrased form) on the tracking sheet and post on the wall of the
audit conference room to communicate the finding to the Center and rest
of the Audit Team. Identification and communication of findings in this
way help prevent duplicate findings and the Center from being surprised
by a finding at the last minute. The audit support team will then review
the evaluation sheet to ensure it is clearly written and complies with
format requirements. Any corrections or revisions are made as necessary,
and the evaluation sheet is typed into the draft report file. If
significant changes are warranted, audit support staff will discuss them
with the auditor. The sheet shall then be printed and stapled to the
front of the original, handwritten sheet. The sheet shall then be placed
in the bin labeled "Auditor."

5\. Auditors should periodically check the Auditor bin after they have
submitted an evaluation sheet to typing to see if it has been typed.
Typed evaluation sheets should be reviewed to ensure that they were
typed up as written and/or to ensure that any changes made by the
support team do not change the intent of the auditor. If changes are
necessary, they should be marked on the evaluation sheet and resubmitted
to typing. If the evaluation sheet meets the auditor's satisfaction, the
auditor shall sign their initials in the space provided next to
"Reviewer" in the top right corner of the sheet. This indicates that the
auditor agrees with the content of the evaluation sheet. The evaluation
sheet shall then be placed in the bin labeled "Team Leader."

6.  The Team Leader (TL) periodically checks the Team Leader bin for any
    evaluation sheets to be reviewed. The TL reviews evaluation sheets
    to ensure findings are valid and easily understood. Any recommended
    changes may be discussed with the auditor if necessary and
    resubmitted to typing. Once the TL is satisfied with the evaluation
    sheet, the TL initials the blank next to "Team Leader" in the top
    right corner. The sheet shall then be given back to typing to make
    any final changes and for compilation into the draft report.

7.  The final audit cards produced at the conclusion of the audit will
    provide adequate detail and technically accurate information to
    fully explain the finding. However, these should be considered as
    draft as they may contain spelling and grammatical errors.