FAA Form 1320-15 (4-82) Supersedes WA Form 1320-2

Subject: REPAIR STATION INTERNAL Date: 09/27/95 ACNo: 145-5

EVALUATION PROGRAMS Initiated by: AFS-350 Change:

1. PURPOSE.

a. This advisory circular (AC) provides information and guidance material that may be used by

repair station certificate holders, operating under Federal Aviation Regulations (FAR) Part 145, to

design and implement an Internal Evaluation Program. The procedures and practices outlined in this AC

can be applied to all of a repair station's operations.

b. There is no regulatory requirement for an Internal Evaluation Program; however, the Federal

Aviation Administration (FAA, encourages such a program to increase the awareness of management

and all employees of their responsibility to promote continuous compliance with all regulatory

requirements and good operating practices. Establishing the type of program described in this document

is completely voluntary.

2. RELATED READING MATERIAL. A list of additional information that may be useful in

understanding and implementing an Internal Evaluation Program can be found in Appendix 3.

3. BACKGROUND. Although safety in air transportation is a responsibility shared by certificate

holders and the FAA, the ways in which this responsibility is discharged differ. While the FAA monitors

certificate holders' compliance through periodic inspections, the certificate holders clearly have a

superior vantage point in monitoring their own performance and have greater resources to do so than

the FAA. Although the FAA provides standards and guidance and promotes compliance through its

surveillance, enforcement, and other programs, it is the

certificate holders upon whom rests the primary and ultimate responsibility to verify that their operations

are continuously in compliance with all regulatory requirements. Because certificate holders are in the

best position to identify deficiencies and promptly correct them, it is to their advantage to have a

procedure in place whereby they monitor compliance and continuously improve their compliance

posture. Internal reviews will improve the certificate holder's ability to identify and correct compliance

problems in a more timely manner, instead of
AC 145-5 09/27/952

Page 2

implementing solutions. There are many ways to quickly identify and correct problems. One of the

methods that has evolved over the past few years, primarily used by the air carrier industry, is called

internal evaluation.

a. Development of the Internal Evaluation Process. The concept of an internal evaluation, as

defined by the FAA, was developed in the late 1980's as part of several FAA initiatives to improve the

partnership between the FAA and the air carrier industry. The FAA had found that air carrier top

management had been shifting most of its attention to financial matters at the expense of operational

issues, particularly compliance with the FAR.

(1) At that time, the FAA considered making an Internal Evaluation Program a regulatory

requirement. This contemplated regulation would have instituted a process whereby an air carrier's

management would be more directly responsible for not only complying with the FAR, but also having a

system to continuously verify that operations were in compliance with the FAR, instead of relying on the

FAA to do such verification through its surveillance. The FAA decided it was premature to develop a

regulatory requirement. However, the FAA considered the concept to be sound and believed that

voluntary Internal Evaluation Programs should be strongly encouraged.

(2) Encouragement came in the form of meetings with industry representatives to explain the

concept and discuss how the FAA could support such a voluntary program. In February 1992, the

FAA published the Air Carrier Internal Evaluation Model Program Guide to provide additional guidance

to the air carrier industry and to FAA inspectors on the implementation of Internal Evaluation Programs

for different types of air carrier operations.

b. Related Programs. The Internal Evaluation Program is closely related to two other recent

programs of interest to many certificated repair stations.

(1) The first program is the FAA voluntary disclosure policy. Under this policy, implemented

for repair stations on May 20, 1994, if a repair station finds a potential violation of the FAR, reports it

to the FAA, and meets other specific conditions, the FAA will not pursue an enforcement action.

(i) An internal Evaluation Program is an effective method for discovering potential violations

and developing corrective actions that are aimed at permanently eliminating the underlying cause of a

problem. Corrective actions of this type are one of the requirements of the voluntary disclosure

program.

(ii) Even though the internal evaluation process supports voluntary disclosures, it is not

necessary to have an Internal Evaluation Program to participate in the voluntary disclosure program, nor

is it necessary to voluntarily disclose any findings from an internal evaluation audit. Although the two

programs work well together, they can be mutually exclusive.

(2) The second related program is the Quality Monitoring (QM) system required by the, Joint

Aviation Authorities (JAA) of any repair stations that perform work on aircraft registered in the member

countries of the JAA that are used in commercial operations.
09/27/95 AC 145-5

Page 3

(i) The JAA was formed in 1970 by the 13 countries of the European Civil Aviation

Conference (ECAC) to facilitate certification of aircraft and aviation products built in Europe. Since that

time, the JAA has expanded its interests to include the safety of aircraft, in particular their design,

manufacture, continued airworthiness, maintenance, and operation. The JAA has developed its own

repair station requirements (JAR 145) that incorporate the concept of a QM function.

(ii) The QM function, as defined by the JAA, includes independent, regular audits and

emphasizes the role of an accountable manager. Both of these are also basic characteristics of an

Internal Evaluation Program. The Internal Evaluation Program includes the requirements of the QM

process and several more features, going well beyond the JAA quality monitoring requirements.

Consequently, a properly implemented Internal Evaluation Program usually will satisfy the JAA

requirement for quality monitoring.

c. FAA Compliance Inspections Results.

(1) The FAA field offices regularly conduct inspections of repair stations as a part of their

surveillance work programs. Additionally, the FAA periodically conducts major inspections of these

facilities through the National Aviation Safety Inspection Program (NASIP) and Regional Aviation

Safety Inspection Program (RASIP). Results of these inspections and routine surveillance indicate that

there are recurring areas of noncompliance that would not have occurred as frequently, or at all, if

Internal Evaluation Programs had been implemented. These areas are:

(i) Failure to maintain a current Inspection Procedures Manual that describes all processes

permitted under the repair station's air agency certificate (§ 145.45(f)).

(ii) Not performing repairs in accordance with manufacturers' procedures and the

Inspection Procedures Manual (H 145.57(a) and 43.13(a)).

(iii) Inadequate or outdated personnel records 145.39 and 145.43).

(iv) Performing work for which the repair station is not authorized (§ 145.53).

(v) Improper handling of parts and materials (H 145.43(d)) and 145.59(a)).

(2) While an Internal Evaluation Program will not eliminate all instances of noncompliance with

the regulations, it will provide the certificate holder with a better overview of the organization, systems,

and procedures that may, in fact, reveal systemic weaknesses or a lack of sufficient management

controls. These weaknesses represent the potential for noncompliance; consequently, identifying them in

a systematic manner will often permit corrective action before a noncompliance situation occurs .

4. DEFINITIONS. The following definitions apply to the discussion in this AC and may not be the

same as similar terms used in other documents or applications.

a. Finding. A conclusion, supported by evidence that there has been or is a process or product

that is not in compliance with an established standard.
AC 145-5 09/27/954

Page 4

b. Concern. A conclusion concerning a system or process that identifies a condition that may

become a finding or a system weakness, which could be the underlying cause of a future noncompliance

situation. A concern is not, at the time of the examination, an example of noncompliance with a

standard. If so, it would be a finding.

c. Observation. A noteworthy feature of a system or procedure. The feature noted is usually a

positive or commendable aspect that should be brought to the attention of management to ensure that

the feature is preserved and perhaps adopted in other places, if appropriate.

d. Recommendation. A specific suggested action that addresses a deficiency (either a finding or a

concern). A recommendation from an internal evaluation audit team should be analyzed to determine if it

should be adopted. Correction of deficiencies should be the responsibility of the affected organization.

e. Control. A key procedure, responsibility, or decision-making position within an organization or

functional area. Effective controls reduce the probability of action contrary to established requirements

and reduce the adverse consequences of such actions should they occur. It is the controls used in all

phases of the repair station's operations that should receive the majority of the attention from the internal

evaluation.

f. Inspection. A critical examination of an event or object for conformance with a standard.

Results of inspections are reported only in terms of findings.

g. Audit. A methodical, planned examination of processes, records, and transactions to verify

compliance with a specific audit criterion. Inspections are normally a part of an audit. Results are

primarily presented in terms of findings and concerns. Auditing measures against a defined standard, but

does not analyze the standard.

h. Evaluation. An examination and evaluation of all aspects of a process, both positive and

negative, and includes a review of standards, if appropriate. An evaluation is a process designed to take

a larger view of an operation or organization than an audit. An audit is a subset of an evaluation. Results

of evaluations are presented in terms of findings, concerns, observations, and recommendations.

i. Internal Evaluation. A comprehensive, continual monitoring process that is initiated and

managed by the repair station, even though the personnel actually conducting various audits as part of

the evaluation program may be internal or external to the repair station. The objective of this process is

to promote attitudes and procedures that build quality into a product rather than depending on

corrections of deficiencies to meet quality goals.

j. Top Management. A certificate holder's chief executive officer (CEO), chief operating officer

(COO), president, or a person in an equivalent position who has the authority to resolve issues and take

action and can be held accountable for quality issues. The FAA believes that top management should be

well aware of the plans, results (findings, concerns, and observations), and follow-up actions undertaken

in an Internal Evaluation Program.
09/27/95 AC 145-5

Page 5

5. PROGRAM FEATURES. Internal evaluations should be part of an overall Internal Evaluation

Program that has written descriptions of the key elements of the program. The program should have a

structured and planned series of evaluations that are designed to improve the quality of all steps and

functions in the process that leads to a final repair station product.

a. Understanding the Basic Features. An Internal Evaluation Program is:

(1) A Systems-oriented Process. A continual process that incorporates the techniques of

inspections, audits, and evaluations to assess the adequacy of managerial controls and processes in

critical systems and to continuously improve those systems based upon the results of regular evaluations.

(2) More than an Audit. A review that extends beyond regulatory compliance to determine the

causes of deficiencies and detect needed enhancements to company operating practices before

deficiencies occur. An Internal Evaluation Program would normally verify compliance with company

procedures, policies, and sound business practices as well as legal requirements.

(3) A Corrective Process. A process that analyzes deficiencies, develops corrective action

plans to correct deficiencies, and performs follow-up evaluations to verify that those corrections have

taken place and have been effective.

(4) A Program with High Level Attention. An independent process that organizationally has

straight-line reporting responsibility to top management.

b. An Additional Quality Assurance Process. The Internal Evaluation Program should not be

misunderstood as a program that replaces existing regulatory auditing requirements, such as the

inspection programs cited in § 145.45 of the FAR. It is comprehensive and includes identifying

corrective actions, verifying that those actions have taken place, and ensuring that problems do not reoccur. Further, one of the most critical aspects of an Internal Evaluation Program is the regular day-today involvement of top management, which typically distinguishes it from the normal repair station

inspection functions.

c. Broad Applicability. The internal evaluation should cover all systems, processes, and products

that are basic parts of the repair station's activities. There is no set list of items to be covered since each

operation is unique, but a representative list of areas to evaluate would include:

(1) Facilities and equipment.

(2) Station authority and limitations versus actual practice including controls over any deviation

authority.

(3) Personnel qualifications, training, and staffing levels.

(4) Manuals and airworthiness data.

(5) Continuity of work and supervision during personnel changes.
AC 145-5 09/27/956

Page 6

(6) Supplier selection, approval, and surveillance

(7) Parts and materials handling.

(8) Inspection and quality control processes.

(9) Tool adequacy and calibration.

(10) Maintenance release process.

(11) Defect reporting.

(12) Records and recordkeeping procedures.

d. Potential Benefits. An effective Internal Evaluation Program may bring benefits to repair

stations that go beyond regulatory compliance. The discipline, structure, and added oversight afforded

an organization through internal evaluation can provide better management information and controls,

leading to increased efficiency.

6. INTERNAL EVALUATION PROGRAM DESCRIPTION. The following are essential

elements of an Internal Evaluation Program. Each of these should be described in a program document.

a. Independent and Well defined Responsibility. As a part of identifying internal evaluation

responsibility and independence, certificate holders should identify resources and personnel to conduct

the Internal Evaluation Program, and they should describe their organizational independence within the

company in light of their internal evaluation functions.

Individuals conducting internal evaluations should not be responsible for accomplishing or managing

work in the areas being evaluated or the tasks being reviewed. This concept may have to be modified

for some very small operations.

(1) For some certificate holders, the size of the operation may justify the costs associated with

having full-time, dedicated resources and personnel in a separate Internal Evaluation Department.

However, when full-time, dedicated resources and personnel are not practical, procedures should be

established so that any persons having direct responsibility for the areas to be evaluated are not involved

in the selection of the evaluation team.

(2) At very small operations, a reasonable Internal Evaluation Program might consist of

developing checklists and a schedule (monthly, quarterly, semiannual, or annual) for accomplishing

checklist items. Even in such cases, the review should include a written statement acknowledging the

completion of the checklist items and the signature of a top management official. Under these conditions,

occasional independent oversight of checklist item development and accomplishment should be

considered.
09/27/95 AC 145-5

Page 7

(3) Certificate holders that use outside resources in support of, or to accomplish an Internal

Evaluation Program should show that use of those outside resources is coordinated through a chain of

command that reflects independence and contact with top management.

(4) A certificate holder's Internal Evaluation Program document should identify the person

and/or group within the organization who has the responsibility and authority to:

(i) Perform or cause to be performed evaluations, audits, and inspections as a part of an

ongoing Internal Evaluation Program.

(ii) Identify and record any findings or concerns and the evidence necessary to substantiate

those findings or concerns.

(iii) Recommend or assist with the development ofsolutions to findings or concerns.

(iv) Verify the implementation of solutions consistent with the action plan and validate that

solutions work.

(v) Communicate and coordinate activities with FAA personnel on a regular basis.

(5) A senior level management representative should accept the responsibility to ensure that an

Internal Evaluation Program is properly established, implemented, and maintained. Further, this

individual should have the authority to see that appropriate corrective actions are implemented in a

timely manner. This management position should be above the level that directly supervises work

accomplishment or procedural development and should have direct contact with the CEO or equivalent.

For smaller certificate holders, it is appropriate that this person be the CEO, president, or equivalent.

This senior level management representative would also meet the requirements for an "accountable

manager" as defined by the JAA.

b. Top Management Involvement and Accountability. To be effective, an Internal Evaluation

Program must have the attention of top management regularly, not just when the program is initiated.

Top management should review internal evaluation results to verify that satisfactory corrective actions

have been implemented.

c. Continual, Structured Process. In order to effectively anticipate potential problem areas and

correct them before actual findings occur, an Internal Evaluation Program should be a continual, ongoing

function. An internal evaluation is intended to be more than spot-check inspections of operating

practices. Of and by themselves, spot-check inspections will do little more than identify symptoms of

potential problems.

(1) A continual process is needed to verify whether findings are isolated instances or actual

symptoms of policy, procedural, or managerial problems. A continual program should include scheduled

evaluations, follow-up evaluations as necessary, and special evaluations when trends are identified.
AC 145-5 09/27/958

Page 8

(2) Having a well-structured program ensures that all areas of operations are covered at

appropriate intervals. It also institutionalizes the process so that a change in personnel does not

adversely affect the program.

d. A Plan for Scheduling Evaluations. It is essential for a repair station's Internal Evaluation

Program to include a defined schedule of activities. This planned schedule will serve to verify that the

Internal Evaluation Program is comprehensive, well controlled, and timely. A published schedule also

provides a vehicle for keeping management informed.

(1) A proper internal evaluation schedule should include a planned cycle for periodically

reviewing areas specified to be covered by the certificate holder's Internal Evaluation Program.

However, the scheduling process should also be dynamic and allow for special evaluations when trends

are identified. In addition, follow-up evaluations should be scheduled as necessary to verify that

corrective action commitments were met and that they were effective in eliminating any reported findings

or concerns. Planned, special, and follow-up evaluations, all of which comprise an effective internal

evaluation schedule, are further described below.

(i) Planned Cycle. An effective program will establish a schedule of events that will be

performed during a set calendar period under the Internal Evaluation Program. All key areas will be

reviewed at least once each year. It is helpful to divide the complete schedule into segments that are

practical from the workload standpoint. However, it is important to schedule evaluations to allow

enough flexibility for resources to be committed to special evaluations or follow-up evaluations, if

needed.

(ii) Special Evaluations. Conduct special evaluations based on concerns or priorities set by

top management. The need for special evaluations can be driven by such factors as a review of industry

trends, FAA concerns, or identified internal adverse trends.

(iii) Follow-up Evaluations. Schedule and conduct follow-up evaluations to ensure that

corrective action commitments were met, to verify that corrective actions (described below) were

effective, and to review steps taken as a result of FAA surveillance findings.

e. Corrective Action Plans. An Internal Evaluation Program should include procedures that

ensure that corrective action plans are developed in response to findings or concerns and for monitoring

corrective action plans to verify their timely and effective implementation. Internal evaluation personnel

should participate in the development of corrective action plans. However, organizational responsibility

and accountability for the development and implementation of corrective action plans should reside with

the technical departments cited in the finding or concern.

(1) A proper corrective action plan should include the following elements:

(i) A detailed description of the finding or concern and how it was discovered. This should

include discussion of the scope and extent of the problem so that candidate solutions can be analyzed

properly.
09/27/95 AC 145-5

Page 9

(ii) Analysis of evidence to determine the root cause(s) of the finding or concern.

(iii) Identification of planned corrective steps to be taken in response to the finding or

concern.

(iv) Implementation schedule, including a time frame for putting corrective steps in place.

(v) The individual responsible for implementing each of the corrective steps.

(2) The individuals responsible for managing an Internal Evaluation Program should facilitate the

corrective action process by:

(i) Ensuring that corrective action plans are developed in a timely manner.

(ii) Verifying that corrective action plans include the elements outlined above.

(iii) Monitoring implementation of corrective actions.

(iv) Providing top management with an independent assessment of corrective action plan

development and implementation.

(v) Initiating scheduled and/or unannounced follow-up evaluations to ensure the

effectiveness of corrective steps specified in corrective action plans.

f. Records. The results of an internal evaluation audit and the review of internal evaluation

information by top management should be documented in reports and other appropriate records,

consistent with the process of internal reporting at the repair station. The repair station should decide

upon the frequency, format, and structure for informing top management of internal evaluation plans,

results, and follow-up actions. It is recommended that the reporting structure also be documented by the

repair station and become a part of its program plan.

(1) Internal Evaluation Program files should include: scheduled evaluation reports; special

evaluation reports, including the trends or other reasons for scheduling a special evaluation; corrective

action plans; and results of follow-up evaluations.

(2) Recognizing that much of the information contained in Internal Evaluation Program records

could be proprietary, a repair station should maintain and secure these records on their premises. If

given to the FAA, proprietary information will be protected by the FAA in accordance with applicable

laws and regulations.

(3) Sharing Internal Evaluation Program information with the FAA at the repair station offices

can enhance the working relationship between the FAA and the repair station. Information not required

by regulation that remains on the repair station's property would not normally be subject to public

disclosure.
AC 145-5 09/27/9510

Page 10

g. Specialized Training and Experience for Evaluators. The evaluators that are used by the

repair station should have training in the specific procedures used by the repair station.

(1) Evaluators should be trained and experienced specifically with the evaluation of quality

systems. General inspection experience only is usually insufficient.

(2) The internal evaluation concept emphasizes the analysis of procedures, systems, and

management controls. It is not another layer of transaction examination and hardware inspections.

Rather than just verifying the work of the inspectors, for example; it analyzes the inspection system.

Consequently, it requires different skills.

7. AUDITING PROCEDURES - EXAMPLE. The basic tool for accomplishing an evaluation is

the audit. An audit provides assurances that an organization's methods conform to its requirements, and

it assesses the effectiveness of management controls. Audits help answer the question, "Are the actual

practices consistent with the defined standards?" The following example demonstrates internal evaluation

procedures and the use of auditing concepts in the context of a typical repair station function. This

example addresses the procedures and systems for ensuring that all parts used in modifications or

repairs are approved for such use. Additional information on auditing techniques is provided in

Appendix 2. The example demonstrates the need for an evaluator to understand the complete system

and the applicable requirements. It is important to remember that the use of the word "system" in this

discussion refers to all elements that impact a function-the relevant policies, procedures, management,

operating personnel, training, paperwork, facilities, and equipment.

a. Background. For the purposes of this example, assume that a repair station has an Internal

Evaluation Program in place and there are trained evaluators available, either within the company or

from outside sources. As part of an ongoing series of audits, a small team of evaluators has initiated a

review of the systems that the repair station uses to control its parts supply and ensure that only correct

(approved) parts are used in any of its work. The objective of this internal evaluation task is to verify

that the repair station has the proper procedures, personnel are properly trained, and the management

controls are functioning so that unapproved parts will not be used. It is not simply a spot inspection of

parts and/or the related paperwork to see if any unapproved parts can be found.

b. Process. For most internal evaluation tasks, including this example, the basic steps are as

follows.

(1) Step 1: Understanding the System. The processes that could affect parts usage in this

repair station include the establishment of policy, the documentation of procedures, the ordering system,

the receiving inspection process, parts storage and tracking, parts selection by the mechanic, and

feedback on the performance of parts. Each of these processes should be reviewed by internal

evaluators. To illustrate the concept, the parts ordering and receiving inspection processes will be

discussed. The types of questions that should be asked to help understand this system are illustrated

below. Note that this is not a comprehensive list, but only examples of the types of questions that should

be on the checklist. For the most part, the questions are oriented toward systems, procedures, and

responsibilities rather than toward the inspection of parts.
09/27/95 AC 145-5

Page 11

(i) What written policy exists? Who is responsible for developing that policy? How is

complete dissemination of the policy ensured?

(ii) Where are the purchasing procedures found? Who is responsible for them and for

seeing that they are followed? Who is responsible for parts selection when ordering? Are automatic

reorder quantities periodically audited?

(iii) How does the repair station identify approved sources of parts? Is there some form of

supplier surveillance? How are rotables tracked when they are at another repair station?

(iv) What are the receiving inspection procedures with regard to parts? Who is responsible,

for these procedures? How are the procedures updated when necessary?

(v) How do receiving inspections personnel know about the policy and procedures? If

there is a change, what is in place to be sure the change is known by all inspectors?

(vi) How are parts stored and traced in the inventory? If there is a shelf life, how is it

tracked? What happens when a part in stock exceeds its shelf life?

(vii) What is the system/procedures for a mechanic to draw a part from stock? Does this

system protect against the selection of an incorrect or unapproved part? Does it work, based on

personnel perceptions?

(2) Step 2: Identifying Controls. Once the evaluators have developed a good understanding of

how the system operates, the next step is to identify the built-in controls that ensure the system functions

in accordance with the repair station's inspection procedures manual (IPM) and the FAR.

(i) Does a schedule exist to conduct periodic reviews of the repair station's policy to

ensure that it reflects the actual functions of the repair station? When an item is added or deleted from

the repair station's capabilities, what procedures are in place to ensure company documentation is

updated?

(ii) Do purchase orders above a certain dollar amount require a higher level of

management's approval?

(iii) How do purchasers know whether a source of parts is approved? Does the repair

station have an approved vendor list, and is it periodically updated? Who is responsible for the list?

(iv) Do the shipping and receiving facilities provide segregated areas for incoming and

outgoing serviceable and unserviceable parts?

(v) Do inspection personnel receive training on receiving inspection procedures? Is the

training documented? Do new inspection personnel receive indoctrination training before conducting

receiving inspections?
AC 145-5 09/27/9512

Page 12

(vi) When inspection procedures are revised, do inspectors complete a record that

indicates they have been made aware of the revision? Is this record reviewed to ensure all inspectors

have received the appropriate information?

(vii) Are periodic parts room inventories conducted to ensure that the inventory tracking

system reflects the actual quantities on hand?

(viii) Are shelf-life sensitive items tracked separately in the computer or other tracking

system? Do parts room personnel conduct periodic reviews of shelf life items and discard any expired

items? Is it clear who is responsible? Is a report that documents the completion of the shelf life review

provided to management? What happens if a shelf-life item is found expired?

(ix) Is access to the parts room restricted to authorized personnel or may anyone enter?

How is access controlled? Who is responsible for ensuring that access is controlled?

(3) Step 3: Evaluation Controls - Do they work? An evaluation of whether a repair station's

controls are properly functioning may require a review of records, inventory lists, parts documentation,

warranty claim records, and an inspection of a sample of parts. Also, an important part of any

verification process is the personnel interviews. If personnel do not know policies or feel that they are

ignored, management controls are not functioning properly. The following types of questions may be

used to conduct this portion of the evaluation.

(i) Does a record exist that indicates that a periodic policy review has been conducted? Is

it complete and up to date? Does a review of the repair station's procedures reconcile against the

current IPM?

(ii) Do high-dollar purchase orders indicate that management approved the purchase with a

signature, initials, etc.?

(iii) Do records list how many parts have been returned to vendors because they do not

pass the receiving inspection? Does the current approved vendor list include a revision date?

(iv) Do training records and interviews indicate that initial training and receiving inspection

training have been conducted for inspection personnel? Do records indicate that training was conducted

within a reasonable time after each revision of the IPM?

(v) How often are differences discovered between the inventory system and the actual

number of parts on hand? Also, how often are stocks depleted without making a new order? How often

must the repair station order items on an aircraft-on-ground (AOG) status?

(vi) When conducting the shelf-life inventory, how often are expired items found? Do

inventory records indicate that these items have been removed from stock and properly disposed of? If

items were found that were substantially beyond limits, is there any documentation of what action was

taken?
09/27/95 AC 145-5

Page 13

(vii) How many warranty claims has the repair station been required to adjust in the past 6

months? How many FAA enforcement actions or letters of investigation has the repair station

experienced in the past 6 months?

(viii) Select a sample of parts using statistical sampling procedures (See Appendix 1). Are

the parts properly tagged or otherwise identified? What paperwork is available to demonstrate that the

part is approved?

(4) Step 4: Reporting the Results. Results of an audit generally do not cover positive items nor

do they recommend actions. Under an Internal Evaluation Program these, along with findings and

concerns, are important elements of the report. Consequently, an evaluation report should include at

least the following.

(i) Scope of the evaluation. This should include the areas evaluated, personnel interviewed

(to be done in general terms to provide management an indication as to the scope and depth of the

review without violating any confidentiality), records examined, sampling plans, etc.

(ii) Results. Descriptions of each finding or observation presented in such a manner as to

indicate the relative importance of each. This will allow responsible personnel to set priorities for

developing responses.

(iii) Recommended corrective actions. In some cases, there may be more than one

alternative recommended; in others, there may not be sufficient information for the evaluators to develop

a recommendation.

(iv) Positive results. (Some might be shared between different units within the repair

station.)

(v) Internal Evaluation Report Form. An AC 145-5 evaluation report standard form

should be established. A possible format is shown in paragraph 7c.

(5) Step 5: Developing Corrective Action Plans. Corrective action plans should be developed

principally by the organization responsible for implementing the corrective action; however, if the

evaluation team has properly conducted its evaluation, it will have a detailed understanding of the

systems and procedures underlying the problems and should be able to assist with the analysis of

alternatives. The internal evaluation team should ensure that a corrective action plan is developed in a

timely manner and includes all the key elements, particularly when corrective action is to be implemented

and who is responsible for implementation.

(6) Step 6: Follow-up Evaluations. To be effective, the Internal Evaluation Program should

have follow-up evaluations any time a significant corrective action is planned. The purpose is two-fold:

to confirm that the action has taken place as planned and to verify that the fix has been effective. If a

properly implemented corrective action does not work, new alternatives should be developed as soon

as possible. Keeping management aware of the results of follow-up evaluations is an essential part of the

program.
AC 145-5 09/27/9514

Page 14
09/27/95 AC 145-5

Page 15

c. Internal Evaluation Report - Example

-- .... ......................

8. PROGRAM PLAN. The FAA recommends documenting Internal Evaluation Program procedures

and responsibilities in a program plan. This paragraph provides suggestions for preparing and structuring

a program plan.

a. Planning Process.

(1) Certificate holders should determine what level of formal program plan development is

consistent with the size and complexity of their operation.

(2) A program plan should describe the duties, AC 145-5 responsibilities, procedures, and

organization of a certificate holder's Internal Evaluation Program. Terms and elements defined in

program plans should be consistent with those outlined in paragraph 7.

Date: September 30, 1994

Evaluation Area: Control of Parts.

Evaluation Objective: Evaluate the procedures and systems for ensuring that all parts used in

modification or repairs are approved for such use.

Evaluator: J. Doe

Method: Management controls identified per attached checklist; 10 percent sample of parts in the

stock room and on the shop floor.

Findings/Concerns: Findings: No significant findings. Isolated discrepancy resolved.

Concerns: The separation of serviceable and non-serviceable parts is not always clear. Although the

tagging procedures were correct for the sample of parts reviewed, additional physical separation would

increase the controls.

Proposed Actions: Reorganize the parts room to establish a separate area for the parts that are not

serviceable. Consider separating those parts into those that are beyond economic repair and those

awaiting repair or overhaul.

Follow-up Recommended: Y X N ____, within 6 months after the evaluation.

Report Routing: Director Material, Chief Inspector, Internal Evaluation File.
AC 145-5 09/27/9516

Page 16

(3) Copies of the program plan should be distributed to appropriate company personnel so that

they are aware of Internal Evaluation Program procedures. In addition, revisions should be made as

necessary to ensure that the program plan continues to reflect the certificate holder's current internal

evaluation procedures and organization.

(4) Even if a certificate holder elects to prepare a written program plan or revise an existing

plan, the FAA will not approve or otherwise formally accept program plan contents. Documenting the

procedures and responsibilities associated with any program is considered good practice. For certificate

holders who choose to prepare a program plan, the FAA will be available to provide assistance if

requested.

b. Plan Structure. A sample outline of a program plan, using the program elements discussed in

this AC, is provided in Appendix A. This outline should be viewed as a sample of items that warrant

consideration when a certificate holder is designing an Internal Evaluation Program. The number of items

addressed and how they are documented will ultimately depend on the complexity of the operation.

9. ORGANIZATIONAL MODELS. Each repair station is unique with regard to size, ratings,

facilities, personnel resources, and method of operation. Therefore, no single type of structure will be

appropriate for all types of internal evaluation organizations. Three basic types that may be used are a

Dedicated Internal Evaluation Department, Dedicated Individual Manager and Temporary Evaluators,

and a Combination of Internal and External Resources. Other organizational structures may also be

appropriate. The most critical elements are an adequate level of independence and a reporting process

that ensures an accountable manager is aware of the evaluation results. It should be emphasized that any

of those organizations must have the support of the highest levels of the company to ensure the

effectiveness of the internal evaluation process.

a. Dedicated Internal Evaluation Department. This model presupposes a large organization with

multiple departments and perhaps, satellite locations. This department is responsible for performing

periodic, comprehensive evaluations of the entire organization. The department would report directly to

the CEO or COO of the repair station as a staff function, yet work closely with the operating

departments to remain aware of current operating practices and to provide constant feedback to those

departments.

PRESIDENT

INTERNAL

EVALUATION

PRODUCTION SCHEDULING INSPECTION MATERIAL

TECH. PUBS.
09/27/95 AC 145-5

Page 17

Figure 1: Internal Evaluation Program Model, Dedicated Internal Evaluation Department

b. Dedicated Individual Manager and Temporary Evaluators. This organizational model has

only one individual fully dedicated to the internal evaluation process. Employees from the various

operating departments, usually one from each department, are used as temporary evaluators. This

method conserves personnel resources, yet ensures a continuing Internal Evaluation Program using

qualified personnel on a part-time basis. The selected individuals report to the head of internal evaluation

only for the time they participate in the internal evaluation process. This model is suited to the

medium-sized company where a full complement of dedicated internal evaluation personnel may not be

justified.

Figure 2: Internal Evaluation Program Model, Dedicated Individual Manager and Temporary

Evaluators

PRESIDENT

PRODUCTION

MANAGER

SCHEDULING

MANAGER

CHIEF

INSPECTOR

MATERIAL

TECH. PUBS.

MANAGER

INTERNAL

EVALUATION

MANAGER

PART TIME EVALUATION TEAM
AC 145-5 09/27/9518

Page 18

c. Combination of Internal and External Resources. This model combines the use of internal

and external resources to create a hybrid Internal Evaluation Program. At least one company person is

designated as the head of the internal evaluation function either on a full-time or part-time basis. This

individual is supported by a combination of company, personnel and external experts in the field to

perform periodic comprehensive evaluations on the organization. The external experts may come from

an industry association, a parent company or a contracted organization, however, the primary internal

resources remain responsible for the results of the evaluations, sharing of information with the FAA,

incorporating needed changes to their ongoing duties, and the organization's basic integrity.

Figure 3: Internal Evaluation Program Model, Combination of Internal and External

Resources

10. FAA RESOURCES AND ASSISTANCE. Although the development and implementation of

repair station Internal Evaluation Programs is completely voluntary, the FAA can provide assistance.

The assistance is available for developing, implementing, and continuing operation of the program. This

section summarizes the FAA resources that are available.

a. Each resource, whether at the field, regional, or headquarters level, should first be approached

through the certificate holder's principal inspector. Availability of some resources, particularly those of

an automated nature, may be limited because of internal FAA requirements. Coordination through the

principal inspector will ensure that appropriate consideration is given to each request for information or

PRESIDENT

INTERNAL EVALUATION

EXTERNAL RESOURCES

PRODUCTION QUALITY

CONTROL

MATERIAL

TECH. PUBS.
09/27/95 AC 145-5

Page 19

assistance made by the certificate holder. Available FAA resources have been subdivided into three

separate categories: automated, personal, and written.

(1) Automated Systems. Automated systems are computer applications developed to assist

FAA personnel in analyzing and overseeing aviation safety. Some such systems are generally accessible

only to authorized FAA personnel. Accordingly, access is restricted and information requests should be

coordinated through the certificate holder's principal inspector. A brief description of the automated

systems applicable to the internal evaluation process is presented below.

(i) Automated Federal Aviation Regulations System (AFARS). AFARS provides access

to the full text of the Federal Aviation Regulations. It permits the user to locate all applicable FAR

references to a particular subject and to find citation and cross-references within the regulations.

AFARS contains the current edition of the FAR and is continuously updated as the regulations change.

(ii) Accident Incident Data System (AIDS). AIDS collects information on aircraft

accidents and incidents involving U.S.-registered aircraft. Users of the system can query the database

for data concerning certain kinds of aircraft and specific causal factors, such as maintenance actions.

(iii) Service Difficulty Reporting System (SDRS). SDRS collects information relating to the

identification of abnormal, potentially unsafe conditions on aircraft or in components and equipment.

SDRS can be used to identify trends before mechanical failures occur. The database is continuously

updated through reports received principally from maintenance personnel at air carriers and repair

stations.

(iv) Policy System. The Policy System tracks certain types of documents written to clarify

regulations issued by headquarters and regional offices. The system contains information from FAA

handbooks, orders, ACs, medical guideline letters, FAR legal interpretations, preambles to rules, policy

memoranda, and the full text of documents originally contained in the Regulatory Background Reference

Subsystem (RBRS). Use of this system, particularly with the principal inspector, might help resolve

differences in the interpretation of specific regulatory requirements.

(v) Project Tracking and Reporting System (PTRS). The PTRS is used primarily for

agency tracking of field and regional personnel workloads. However, inspectors also enter results of

inspection activities, so it is a source of useful information on technical safety issues as well. Use of the

Voluntary Disclosure Policy will be recorded by the PTRS and thus the associated information will be

available for analysis by other FAA inspectors and, on a de-identified basis, by repair stations.

b. District, Regional, and Headquarters Resources. Listed below are resources available to

assist repair stations with Internal Evaluation Programs. Contact with any of these resources should be

coordinated with the repair station's

principal inspector.

(1) District Level. Flight Standards District Office personnel can work directly with the repair

station and headquarters personnel to implement this program. Headquarters will provide guidance to

the district office upon request.
AC 145-5 09/27/9520

Page 20

(2) Regional Level. Each region provides technical and managerial assistance to field offices

within its region and helps to coordinate activities among the offices. Although the principal inspector

remains the focal point, regional personnel, by virtue of their wide scope of activities, are often able to

provide assistance beyond that available from the principal inspector.

(3) Headquarters Level. Inspectors with questions concerning the internal evaluation program

for repair stations may contact the Flight Standards Aircraft Maintenance Division (AFS-300).

c. Written Resources and Training Courses. The FAA provides technical publications and

educational courses that may be of assistance to repair stations. FAA publications include ACs and

documents written primarily for FAA inspectors that are also helpful to certificate holders. In many

cases, FAA internal publications describe the methods of surveillance used by inspectors and clarify the

standards that apply to a certificate holder. This understanding can be used by the repair station to

improve internal evaluation procedures that address regulatory compliance.

(1) Available written publications are identified in pamphlet FAA-APA-PG-13, entitled Guide

to Federal Aviation Administration Publications. This free publication can be ordered from the following

address:

U.S. Department of Transportation

Property Use and Storage Section

M-48.3

Washington, D.C. 20590

The guide describes all FAA publications, including material directed primarily at Part 145 repair

stations. Some of the available publications are listed in Appendix 3 of this AC.

(2) Additional FAA resources exist at the FAA Academy, located at the Mike Monroney

Aeronautical Center in Oklahoma City, Oklahoma. The Academy provides FAA initial and recurrent

training in a number of technical disciplines. Industry personnel are invited and encouraged to attend

these courses. FAA courses focus on such key issues as the evaluation of aviation management systems,

quality assurance systems analysis review techniques, and aircraft maintenance reliability programs.

(i) Academy courses are open to industry personnel on a space-available basis. The

techniques and procedures learned during FAA courses provide repair station personnel with valuable

information applicable to the analysis of their own operations.

(ii) Repair stations interested in participating in Academy courses should first request a

current copy of the FAA Catalog of Training Courses. Courses listed in the catalog are not necessarily

offered continuously. The catalog and a listing of the courses currently available can be requested from

the address below. Repair stations should make their request on company letterhead. Address the

request to:
09/27/95 AC 145-5

Page 21

Program Director, FAA Academy, AMA-1

P.O. Box 25082

Oklahoma City, Oklahoma 73125

(iii) Once a particular course has been selected, the repair station should contact the

Academy to ascertain space availability and the cost of the course. Questions directed to the Academy

should be on company letterhead and should be sent to the address above.

11. CONCLUSION. Development of Internal Evaluation Programs, as discussed in this AC, should

help the company to ensure that company policies and procedures are responsive to growth and change

and continually comply with appropriate safety requirements. Furthermore, the FAA strongly

encourages certificate holders to make Internal Evaluation Programs an integral part of their everyday

management process and take full advantage of the voluntary disclosure policy. Aviation safety is best

served by programs that allow certificate holders to identify and correct instances of noncompliance and

invest more resources in efforts to preclude their recurrence, rather than pay civil penalties after

noncompliance.

Thomas C. Accardi

Director, Flight Standards Service
AC 145-5 09/27/95

Appendix 1

Page 2

APPENDIX 1. PROGRAM PLAN SAMPLE OUTLINE

The following is an outline of a typical internal evaluation program plan, with brief descriptions of each

main subject area.

1. OBJECTIVE AND PQLICY. The objective should be a statement that clearly defines the

purpose and structure of the repair station's Internal Evaluation Program. Policy statements following the

objective should indicate that the corporate view of the program is that it is independent, that it actively

involves top management, and that it is an ongoing process designed to identify potential problem areas.

2. DEFINITION OF TERMS. Terms that will be used consistently in the Internal Evaluation Program

should be defined. For example, a repair station should have a procedure for categorizing results (e.g., a

finding or concern). If desired, there can also be a scheme for ordering findings so that management can

quickly identify the ones that have the most serious consequences. These categories, as well as other

terms applicable to the internal evaluation function, should be clearly defined and documented so that

company personnel can understand and properly interpret them.

3. DUTIES AND RESPONSIBILITIES. The duties and responsibilities of internal evaluation

personnel should be documented.

a. The repair station should specify which personnel are responsible for performing the following

tasks:

(1) Supervising the internal evaluation function.

(2) Developing evaluation schedules.

(3) Performing evaluations, audits, and inspections as a part of internal evaluation.

(4) Identifying and recording any findings or concerns.

(5) Collecting the evidence necessary to substantiate findings or concerns.

(6) Initiating, recommending, or providing solutions to problems.

(7) Monitoring the development and implementation of corrective action plans.

(8) Maintaining and updating internal evaluation files.

(9) Verifying the implementation of solutions.

(10) Coordinating Internal Evaluation Program activities with FAA personnel.
09/27/95 AC 145-5

Appendix 1

Page 1

b. This section of the program plan should show that personnel responsible for the tasks listed

above are independent of company procedural development and work accomplishment that would be

subject to evaluations. The supervisor of the internal evaluation function should either be from top

management or have straight-line reporting authority to top management.

c. When full-time dedicated resources are not practical, developed procedures should show that

persons having direct responsibility for the areas to be evaluated are not involved in the selection or

supervision of the internal evaluation team nor are they involved in any evaluation functions. In addition,

it is highly desirable to have temporary evaluation personnel exempt from other duties and completely

dedicated to the Internal Evaluation Program.

4. ORGANIZATION CHART. An organization chart that clearly shows the position of the internal

evaluation function in the repair station's organization should be prepared. This position

should reflect both the program's independence within the corporate structure and straight-line reporting

to top management.

5. REPORTING PROCEDURES. Reporting procedures should show that top management will be

informed through straight-line reporting channels about the schedule, plans, results, and follow-up

actions of the Internal Evaluation Program. The procedures outlined in this section of the program plan

should specify the frequency, format, and structure for reporting information to top management. A

procedure should also be developed that explains how the review by top management will be

documented.

6. SPECIFIED AREAS COVERED. A certificate holder should specify the areas that are within the

scope of review under the Internal Evaluation Program. The FAA believes that the most effective

Internal Evaluation Program will encompass a complete review of all of the repair stations technical

operations. However, a repair station may elect to limit the evaluations to only a part of the technical

operations, and it may elect to include nontechnical functions that are not subject to FAR requirements.

Also, the repair station may elect to phase-in an internal evaluation program with limited coverage for an

initial period and expanded coverage in later phases.

7. SCHEDULE PROCESS. The scheduling process should be comprised of the following elements:

a. Scheduled evaluations over a predetermined calendar period.

b. Special evaluations when trends are identified or priorities are set by top management.

c. Follow-up evaluations to verify the effectiveness of corrective action plans or in response to

findings.

d. The program plan should include procedures for planning, developing, and coordinating the

internal evaluation schedule. The responsibility for planning and developing schedule activities should

also be defined.
AC 145-5 09/27/95

Appendix 1

Page 2
09/27/95 AC 145-5

Appendix 1

Page 3

8. RECORDS. The Internal Evaluation Program should have a defined recordkeeping process.

Procedures should specify how records are filed and maintained. Standard forms or formats for filing

reports should also be specified. The following is a suggested list of records that should be maintained:

a. Scheduled evaluation reports.

b. Special evaluation reports.

c. Follow-up evaluation reports.

d. Responses to findings or concerns contained in reports.

e. Corrective action plans submitted in response to findings.

9. TRAINING. If feasible, the repair station should specify that evaluators receive some type of

training in quality auditing and evaluation principles, and systems analysis techniques. This training could

be from any one or combination of the following:

a. In-house training course or on-the-job training.

b. College courses.

c. Home study course materials.

d. Industry seminars and workshops.

09/27/95 AC 145-5

Appendix 2

Page 1

APPENDIX 2. AUDITING TECHNIQUES

1. PURPOSE. This appendix reviews some of the standard auditing techniques that would apply to

evaluations conducted under a repair station Internal Evaluation Program.

2. INTRODUCTION. Every organization establishes plans, objectives, a method for translating

objectives into actions. This is generally accomplished through the development of policies, systems, and

procedures.

a. Policies are decisions; systems and procedures are the methods for carrying out decisions.

Hence, establishing and maintaining effective policies, systems, and procedures are ways of controlling

the activities of an organization.

b. Whatever an organization's functional activities, some method of review or appraisal is necessary

to compare accomplishments with intended policies, systems, and procedures. Examples of poor

products may be the result of a breakdown in procedures,. lack of supervision, or insufficient

management standards.

3. AUDIT PROGRAM. A management audit program is defined by its areas of examination. In any

management audit, the critical areas for review are:

a. Oragnization. How the area under appraisal is organized and how the actual organization

compares to the stated company organization.

b. Policies. What policies exist and how effective are those policies? Such a review should also

include a determination of how policies support compliance with applicable laws and regulations.

c. Systems and Procedures. What systems and procedures exist and what possible defects or

irregularities exist?

d. Controls. What system controls exist and are they adequate and effective?

4. SYSTEMATIC AUDIT PROCEDURES. A systematic audit method is based upon accepted

auditing practices, so that those relying on the auditing program can be confident that results are

accurate.

a. The main steps in auditing are the planning, implementation, and reporting phases. In the planning

stage, the scope and objectives of an audit are determined by the needs of the client (in conjunction with

auditors) along with the timing for the initiation and completion of the audit. Auditors are selected based

on individual qualifications and the scope of the task. Work assignments for the auditors are determined.

Entities within an organization are notified of a pending audit. The audit team makes a preliminary

assessment of the established system (controls, procedures, systems, documentary evidence, etc.)
AC 145-5 09/27/95

Appendix 2

Page 2

within the area to be audited. Checklists for the audit area are developed and a schedule for conducting

the audit is determined.

b. Evidence is gathered via review of documents, observations, records checks, and interviews

with key individuals in the areas under review. Detailed checklists are used to ensure that all areas of

interest are covered. Auditors are careful to keep a record of auditor activities, including the times and

dates for completing an audit activity. In that way, audit results can generally be reproduced, and the

status of work completed and still to be accomplished is constantly known.

c. The evidence developed during an audit may be portrayed as findings or concerns. Upon

completion of an audit, results are presented to relevant management before a written report is

developed.

d. In the reporting phase of the audit, a written report on audit results is prepared, including

information that can help the user of the report to take appropriate corrective actions. A written report

generally will include a description of the purpose and scope of the audit, details of the audit activity,

audit results, and supporting evidence.

5. APPLYING AUDITING TECHNIQUES TO INTERNAL EVALUATION. The following

discussion provides an overview of the four fundamental steps suggested when conducting an audit

under an internal evaluation program.

a. Identify Key System Characteristics: Understanding the Requirements. As a first step, an

evaluator preparing an audit program needs to be cognizant of external requirements and other factors

that may impact management's decisions. obvious examples of such external influences within the

aviation environment include:

(1) Regulations.

(2) FAA guidance and policy.

(3) Equipment manufacturer guidance and specifications.

(4) Industry trends or incidents.

(5) An effective audit program includes provisions to verify that appropriate external influences

are recognized in the management structure. An evaluator should use current FAA programs and

materials (available through the local Flight Standards District Office) to ensure that the audit program

addresses all appropriate external requirements and other factors.

(6) An internal evaluation should normally go beyond merely verifying compliance with the

regulations and other applicable legal standards. A thorough program will be designed to determine and

evaluate how an organization's policies, practices, systems, and procedures account for and incorporate
09/27/95 AC 145-5

Appendix 2

Page 1

all external and internal requirements. Therefore, the evaluator must know how to translate pertinent

requirements into key system characteristics.
AC 145-5 09/27/95

Appendix 2

Page 4

(7) For example, § 145.43 of the FAR states that a certificate holder must keep certain

records pertaining to supervisory and inspection personnel. A compliance audit would simply ensure

that copies of personnel qualification records are maintained. The results either confirm compliance or

noncompliance at the point and time of the audit, but provide no assurance that compliance will exist

each day unless, of course, such an audit is done every day.

(8) Like a compliance audit, an internal evaluation will also verify that qualification records are

maintained in accordance with § 145.43. However, because internal evaluation is a management audit,

the first step becomes identifying how this external requirement is managed within the personnel

qualification process. In other words, an evaluator should recognize that the recordkeeping system is a

key characteristic of the personnel qualification process.

(9) The personnel qualification process should include a series of controls that monitor and

verify the retention of personnel qualification records. The evaluator needs to verify that management

has designed the personnel qualification process to include such controls. An audit of personnel records

should not begin until the existence of these controls has been verified.

b. Identifying System Controls: Use of the Checklist.

(1) The second step of the internal evaluation process involves translating the understanding of

key system characteristics into a method for identifying system controls, which are inherent management

features that ensure that internal and external requirements are fulfilled.

(2) A key technique for the evaluator to accomplish this step is through the development of a

checklist. On the simplest level, a checklist denotes points to be checked. More importantly, checklists

help evaluators and auditors determine the correct order in which to proceed with an evaluation. A

checklist supplements an evaluator's memory and provides the basis for reconstructing an audit trail.

c. Criteria checklist.

(1) One important form of checklist is the "criteria" checklist. A criteria checklist verifies

compliance with a standard, regulation, or called-for procedure. In essence, a criteria checklist question

is the transposition of a standard, regulation, or procedural requirement into a question.

(2) For example, § 145.45 of the FAR requires stations to prepare and keep current a manual

containing inspection procedures. A criteria checklist question of such a requirement asks, "Does the

certificate holder maintain a current inspection procedures manual?" Standards, requirements, etc., often

have subset standards. A criteria checklist, therefore, needs to address individual subsets of a particular

standard. Each manual required by § 145.45 includes a number of required subsets, such as explaining

the internal inspection system, stating the preliminary inspection procedures, and describing the

continuity of inspection responsibility. Such subsets must be determined and made part of the criteria

checklist when assessing a standard.
09/27/95 AC 145-5

Appendix 2

Page 3

d. Departmental Checklist.

(1) Another type of checklist, sometimes referred to as a "departmental" checklist, structures

questions according to discrete departments and/or tasks. Such checklists define the organization,

resources, equipment (if any), and/or tasks, and ask questions such as, "What is the personnel level

required for this department or tasks?" and, "Are sufficient personnel available to perform the task or

departmental function?"

(2) For example, a "departmental" checklist for the evaluation of a repair station's incoming

materials inspection process will ask questions such as, "Who receives incoming parts and materials?";

"Is the received part invoice compared to the purchase order?"; and, "What procedure ensures that high

priority parts are handled quickly and carefully?"

e. Differences Between the Criteria and Departmental Checklist.

(1) As demonstrated from the above discussion, a departmental checklist goes beyond

verifying that an organization complies with required criteria. A departmental checklist identifies the

existence of controls within a given area to guide the accomplishment of external requirements and to

meet internal plans and objectives.

(2) For example, recordkeeping should be a key characteristic of the personnel qualification

process. A criteria checklist would ask, "Are personnel qualification records kept current?" A

departmental checklist, however, would break down this requirement into a series of questions designed

to identify the existence of controls that will ensure that adequate recordkeeping methods are a part of

the personnel qualification process. For instance, questions might include:

(i) Who is responsible for maintaining personnel qualification records?

(ii) What procedure verifies that personnel qualifications records are maintained for each

supervisor? Are these procedures written?

(3) obtaining answers to these types of questions is a primary part of the internal evaluation

process. They provide indications of how well the organization is managing compliance. The answers

help the evaluator identify areas of potential weakness, and thus determine best how to allocate audit

resources to test the process.

f. Evaluation of Controls - Do They Work?. Only after identifying the existence of key controls

should the evaluator actually inspect, observe, or sample the defined audit population. In this manner,

the evaluator not only verifies current compliance, but also evaluates the effectiveness of identified

controls to ensure continuous compliance.

(1) An essential technique for the evaluator to apply during this step is sampling. A sampling

rate of 100 percent would, of course, contribute to a high degree of reliability and accuracy. Practical
AC 145-5 09/27/95

Appendix 2

Page 4

considerations such as time and economics, however, often obviate the desire to sample every item in a

population. But faced with 1,000 records of an action, how many
09/27/95 AC 145-5

Appendix 2

Page 5

records should an auditor examine before he or she can be reasonably certain that a conclusion

about the records is reliable? Tables to guide sample size selection are readily available.

(2) The table below is an example of those used in sampling. This table is part of the guidance

FAA inspectors receive for conducting inspections. This same table is found in the FAA's Air

Transportation Operations Inspector's Handbook (Order 8400.10).

Table 1. Table of Sample Sizes - Number of Inspections Recommended to Achieve a 95%

Confidence Level

Population of Homogeneous Group Recommended Number of Inspections

Up to 100 50% (50)

200 40% (80)

400 35% (140)

500 33% (165)

1,000 28% (280)

2,000 16% (322)

3,000 11% (330)

4,000 8.8% (352)

5,000 7.7% (355)

10,000 3.7% (370)

(3) Evaluators should recognize that sampling is only one tool used in an audit. The sampling

table above is used as part of a random sampling process. Random sampling of an entire population

may not be the most effective means to check for a parameter. Samples supplement, but do not replace,

common sense, good judgment, and experience. when combined with the results of the checklist,

deficiencies found in the sample can begin to be evaluated as either isolated occurrences or signs of

control/management weaknesses.

g. Reporting Results. As with any audit, the final stage of an audit performed under the internal

evaluation approach should be the concise reporting of results.
09/27/958 AC 145-5

Appendix 3

Page 1

APPENDIX 3. INFORMATION TO ORDER RELATED READING MATERIALS

1. PURPOSE. Related reading materials may be ordered as follows:

a. Air Carrier internal Evaluation Model Program Guide, dated February 1992: $27 per copy.

order from:

National Technical Information Service (NTIS)

U.S. Department of Commerce

5285 Port Royal Road

Springfield, VA 22161

Tel: 800-553-6847 Fax: 703-321-8547

703-487-4650

Reference: PB92238658XSP

b. Joint Aviation Requirements (JAR) Part 145, Approved Maintenance Organizations. order

from:

Civil Aviation Authority

Printing and Publishing Services

37 Gratton Road

Cheltenham, GL502BN

England

Attn: Sales Department

Fax: 44-242-584-139

c. International Organization for Standardization (ISO) Part 9000, Quality Systems: $285,

plus $16.45 handling charge. order from:

American National Standards Institute (ANSI)

11 West 42nd Street

New York, NY 10036

Tel: 212-642-4900 Fax: 212-302-1286

d. FAA National Aviation Safety Inspection Program Annual Reports: $27 per copy. Order from:

National Technical Information Service (NTIS)

U.S. Department of Commerce

5285 Port Royal Road
AC 145-5 09/27/95

Appendix 3

Page 2

Springfield, VA 22161

Tel: 800-553-6847 Fax: 703-321-8547

703-487-4650

Reference: ADA274471XSP

e. AC 145-3, Guide for Developing and Evaluating Repair Station Inspection Procedures

Manuals (2/13/81); and Change 1 (2/10/82): Available free of charge. order from:

U.S. Department of Transportation

Property Use and Storage Section

M-48.3

Washington, DC 20590

Fax: 202-366-2795
US Department

of Transportation

FEDERAL AVIATION

ADMINISTRATION

800 Independence Ave., S.W.

Washington, D.C. 20591

FORWARDING AND RETURN

POSTAGE GUARANTEED

Official Business

Penalty for Private Use $300

BULK MAIL

POSTAGE & FEES PAID

FEDERAL AVIATION

ADMINISTRATION

PERMIT NO. G-44