Page 1 of 12

SMS Pilot Study Gap Analysis

Executive Summary:

Sacramento County Airport System (SCAS) was one of the airports selected by the Federal Aviation

Administration (FAA) to conduct a Safety Management Systems (SMS) pilot program at Sacramento

International Airport (SMF). This document serves as the first deliverable to the FAA under the pilot

program, outlining those areas of the Safety Program benchmarks listed herein where SMF is

capable, emergent or gapped. This is not intended to represent all areas typically found in a fullycapable SMS program, rather it specifically addresses those areas of interest to the FAA. SCAS

engaged a consultant to facilitate the process and ensure rapid delivery (Faith Group, LLC).

Gap Assessment - SMF Assessment Comments

Safety Policy & Objectives Emergent

Policy

The County started a Safety Dept which is dedicated to

SMF approximately 2 years prior. It needs non-punitive

reporting and an explicit commitment statement to

safety that meets AC-150/5200-37

Objectives GAP

Safety Risk Management Gapped

Determine Risk(s) GAP

ID Severity/Probability GAP

Develop Mitigation Strategy GAP

Apply, Track, Monitor

Mitigation Compliance Suite software

Assess & Modify as

Necessary GAP

Safety Assurance Emergent

Performance Indicators GAP

Self-Auditing Part 139 Compliant (Needs SMS auditing process)

Safety Oversight Resources

Commitment exists; needs structure to support SMS

purpose.

Non-Punitive Safety Reporting GAP

Review/Feedback Reporting Part 139 Compliant

Communicate Findings Part 139 Compliant

Promote Safety Integration GAP

Safety Promotion Emergent

Training & Education

Training focused on SIDA/Driver/IIPP; Training has to

expand to include SMS training

Safety Communications Safety Committee monthly meeting

Continuous Improvement Commitment needs to be in writing (to include SMS)
Page 2 of 12

The SCAS has all the safety programmatic elements required by regulation today and has a

demonstrated commitment for developing a leading SMS program. The gapped elements listed

above are in essence, a confirmation that the US is heading into SMS for the first time.
Page 3 of 12

SACRAMENTO COUNTY AIRPORT SYSTEM

SAFETY MANAGEMENT SYSTEM PILOT PROGRAM

GAP ANALYSIS

Introduction:

The FAA published a list of 20 airports eligible to receive Airport Improvement Program Grants (AIP)

to develop pilot SMS programs in a preliminary effort to develop a nationwide standard and requisite

regulation. In addition there are 5 airports participating in the pilot program without AIP grants. The

airports with AIP grants included Sacramento International Airport (SMF) which awarded the

contract to Faith Group, LLC in December of 2007. The SMS team, including members of SMF

staff, airline tenants and stakeholder agencies, such as TSA and FAA, was introduced to the

concept in a Kick-Off Meeting held in January of 2008. Work began immediately and this document

serves as the first FAA deliverable; the Gap Analysis.

Approach:

The FAA published a requirement stating that the SMS Manual and program should “...identify

which elements of the airport operator’s existing practices and guidance materials currently meet

SMS requirements, which elements do not, and how these latter practices and documents will be

revised in the future for consistency with an SMS Plan.”1 The FAA requires a Gap Analysis or

benchmark study to determine gaps in (a) Safety Policy and Objectives, (b) Safety Risk

Management, (c) Safety Assurance and (d) Safety Promotion. These areas were evaluated using

the list of 21 deliverables required by the SMS Pilot. Therefore, the initial Gap Analysis undertook

an assessment of the organizational capabilities at SMF as it pertains to the guiding principals of

SMS. The method of determining the existent capabilities considered a combination of stakeholder

interviews, document search and review, and a preliminary operational review (observation). This

process reviewed over 20 documents, interviewed key staff at SMF and spent several days on the

ground at the airport monitoring ramp operations.

Faith Group uses a proprietary

approach to investigate an airport’s

operating characteristics. This process

considers four key areas; People,

Process, Technology and Training.

These four areas are inseparable and

any program determined to be

successful must ensure it addresses

each of these organizational elements.

In brief, the technology is usually the

simplest element to address;

Technology is cheap and relatively

simple to install. Process is more

difficult, especially if it is addressing new

approaches to a task, using new

technology, but can be simply

documented to ensure it addresses all

1 Office of Airport Safety and Standards, Airport Certification Program, Safety Management Systems (SMS) Pilot Study

Participant’s Guide, April 6, 2007
Page 4 of 12

stakeholder needs. People are the hardest element to identify because of legacy organizational bias

and in some cases, resistance to change. Finally, training is the most neglected element, but

surprisingly the most cost-beneficial of the elements. A dollar invested in effective training will

continue to pay dividends, while technology needs to be replaced every several years.

Using this process, SMF moved forward to identify the key gaps which exist today as they pursue a

fully-capable SMS program. The goal was to complete a gap analysis which will invigorate the

development of the Safety Management Plan using the 21 deliverables listed below.

FAA 21- Pilot Program Mandatory Deliverables

1 - Safety Policy Statement and communication

2 - Airport Safety Goals

3 - Plan for employee SMS indoctrination/training

4 - Process to identify training requirements

5 - Plan to validate training effectiveness

6 - Process to communicate policies/objectives

7 - Plan and description of employee non punitive reporting

8 - Organizational chart of all key personnel

9 - Description safety risk management process (SRM)

10 - Guidance on use of SRM and trend analysis

11 - Process for documenting SRM results

12 - Management follow up on SRM

13 - Description of quality mgmt/risk mgmt

14 - Plan to integrate apron safety into SMS

15 - Method to document self auditing and findings

16 - Method to document self inspection

17 - Plan to integrate SMS program to overall operation

18 - Plan for training, education, communication, competency

19 - Procedures to promote safety awareness

20 - Process to document and review lessons learned

21 - Schedule for implementation and anticipated costs

Gap Analysis Process:

The process involved an in-depth review of existing documentation at the airport, supported by a

vetting process during stakeholder interviews. The team reviewed the following documents at SMF:

139 Inspection Reports

Airport Certification Manual (ACM)

Airport Emergency Plan (AEP)

Aircraft Emergency Response documentation

Sign Layout Plan

Operations Daily Log

Emergency Response Reports

Ops Daily Inspection Reports
Page 5 of 12

Sacramento County Airport System Organizational Chart

Airport Security Plan

Safety Reports

Upon initial completion of the document review, the team met with the following staff personnel to

survey ( see Attachment A) current safety program status:

After completing the survey a results matrix was developed (see Attachment B).

The team also sent questionnaires (survey primers) to other key stakeholders, outside the airport

organization, to gauge the level of SMS activity outside the immediate domain of the airport (see

attached survey form). For example, while the tenant airline SMS capabilities is outside the scope, it

is indeed useful to understand their corporate systems as it pertains to operations on the SMF ramp

and non-movement areas, a key element of this study.

Once the preliminary assessment was completed, a Gap Matrix was developed using the 21 FAA

SMS Pilot Deliverables as a guide. Each was given a subjective “go/no-go” determination, however

further discussion on each topic is provided in order to fully understand the situation which exists at

the airport. For example, while the airport may have a safety policy in place, it may want to consider

amending their overall SCAS Mission Statement to incorporate mention of its safety commitment.
Page 6 of 12

Gap Analysis Findings:

On the outset it was noted that every member of the SMF team was both committed and

enthusiastic in the pursuit of a well-organized SMS program. Actively using the latest technologies

to support training objectives in security and ground vehicle operations, SMF was already taking

steps to provide the highest levels of safety practicable. Clearly, the SMS program will provide a

means of better situational awareness capture on a host of safety issues which affect the airport.

The initial findings in the Gap Matrix are captured in the graph below. Each deliverable was

measured against three subjective descriptors:

Capable: The airport is actively implementing this element.

Emergent: The airport has some elements of this programmatic requirement.

Gapped: The airport does not have evidence of elements of this requirement.

Gap Assessment - SMF Assessment Comments

Safety Policy & Objectives Emergent

Policy

The County started a Safety Dept which is dedicated to

SMF approximately 2 years prior. It needs non-punitive

reporting and an explicit commitment statement to

safety that meets AC-150/5200-37

Objectives GAP

Safety Risk Management Gapped

Determine Risk(s) GAP

ID Severity/Probability GAP

Develop Mitigation Strategy GAP

Apply, Track, Monitor

Mitigation Compliance Suite software

Assess & Modify as

Necessary GAP

Safety Assurance Emergent

Performance Indicators GAP

Self-Auditing Part 139 Compliant (Needs SMS auditing process)

Safety Oversight Resources

Commitment exists; needs structure to support SMS

purpose.

Non-Punitive Safety Reporting GAP

Review/Feedback Reporting Part 139 Compliant

Communicate Findings Part 139 Compliant

Promote Safety Integration GAP

Safety Promotion Emergent

Training & Education

Training focused on SIDA/Driver/IIPP; Training has to

expand to include SMS training

Safety Communications Safety Committee monthly meeting

Continuous Improvement Commitment needs to be in writing (to include SMS)

While these descriptors are subjective in nature, they do offer the potential means for trend

identification. The team used the FAA Advisory Circular 150/5200-37, the FAA SMP Outline, and

IATA SMS as a guideline for making these determinations. The focus of this investigation has been

airport-specific (SCAS non-movement area and personnel). It does not provide the opportunity to
Page 7 of 12

assess tenant safety programs much beyond anecdotal information and observation. While many of

the tenants of SMF have freely offered advice and survey response, there is some resistance to

releasing details regarding their corporate safety programs. The intent is to provide a more detailed

assessment with respect to integrating tenant safety programs with the SMF SMS in the body of the

SMP. It should suffice to say at this point that the range of safety programs associated with various

tenants spans from nearly non-existent for some, to highly sophisticated SMS implementation for

major air carriers operating on the airport. We believe this information will be a key element to

establishing a comprehensive SMS program, and expect to share lessons learned among all tenants

operating at the airport. The promotion of a single integrated SMP would be advocated as it would

assist to achieve standardization and common understanding and implementation of SMS across

the board amongst the airport and its tenants ensuring a bigger safety net.

The focus of this gap analysis, as required by the FAA SMS Pilot Study Participant’s Guide2has

been

 Safety Policy and Objectives

 Safety Risk Management

 Safety Assurance

 Safety Promotion

Safety Policy and Objectives

Generally accepted industry standards promotes Commitment of Senior Management to Implement

SMS; Commitment to continual safety improvement and International Civil Aviation Organization

(ICAO) guidance describe Safety Management Systems in terms of four distinct elements. They

include:

Safety Policy3

Management’s commitment to safety should be formally expressed in a statement of the

organization’s safety policy. This policy should reflect the organization’s safety philosophy and

become the establishment of the SMS. The safety policy outlines the methods and processes that

the organization will use to achieve desired safety outcomes. A safety policy will be signed by Top

Management and will typically contain the following attributes:

 The commitment of senior management to implement SMS

 A commitment to continual safety improvement

 The encouragement of employees to report safety issues without fear of reprisal

 A commitment to provide the necessary safety resources

2 Office of Airport Safety and Standards, Airport Certification Program, Safety Management Systems (SMS) Pilot Study

Participant’s Guide, April 6, 2007

3 AC 150/5200-37, page 3
Page 8 of 12

 A commitment to make safety the highest priority

SMF has an established Safety Policy statement that is essentially compatible with the intent of

SMS, in that it is indeed documented in the County’s Injury and Illness Prevention Program (IIPP)

(see Attachment C). SMF has also established a full time Safety Specialist. The executive

leadership of SCAS is clearly committed to the establishment of SMS and as such is ready to take

the necessary steps to fully implement this program. The policy calls out specific language to its

commitment:

“It is the policy of the Sacramento County Airport System to provide a safe work

environment for all employees, to the extent feasible, which is free of recognized hazards.

The goal of this policy is to eliminate work related illnesses & injuries and to ensure

compliance with CAL-OSHA Rules and Regulations.” 4

The Policy must be expanded to explicitly indicate the company’s intention to commit to safety in

everything that it does – all staff must become aware that their daily work must be conducted in a

safe manner.

While employees appear to be encouraged to work safely, there is no formal non-punitive reporting

program in place and requires more effort to establish such a program. The airport appears to be

committed to providing reasonable resources to support its safety programs, and states such in its

current Safety Policy Statement (see Attachment C). In the initial meetings with Senior

Management, it was made abundantly clear that SCAS is committed to making safety its highest

priority.

Assessment: Emergent (non-punitive reporting gapped)

Required Effort:

1. The Policy Statement should include verbiage to improve best safe practices to prevent

illness, injury as well as preservation of assets.

2. Specific language needs to be included for non-punitive reporting, as well as information or

pointers to more specific direction for the program.

Safety Objectives5

SMS requires the support of senior management. SMS also requires that Top Management in the

organization, one with the authority to adequately control resources, be assigned SMS

responsibilities. In addition to having a basic understanding of the SMS, effective decision-makers

understand how to use SMS outputs as inputs to the SMS lifecycle as described in Figure 2-1.

Executives and managers also understand when safety risk management is necessary, and when to

elevate decisions and the supporting information to a higher level. Some key elements of

accountability within an organization are:

 The organization’s policy concerning responsibility and accountability, including written

guidance regarding the safety authorities and responsibilities of all key personnel assigned to

the airport

4

Sacramento County Airport System, Illness and Injury Prevention Program (IIPP)

5 AC 150/5200-37, page 3
Page 9 of 12

 Identification within the system of someone responsible for administration of the overall SMS.

Often, that one responsible person will be the Safety Manager. This person reports to the

highest level of management to assure appropriate consideration of all reports,

recommendations, and issues

 At larger airports, operations may support the Safety Manager being a full-time permanent

employee and in some cases having a support staff. Some airports may have an existing

risk management office that could substantially meet SMS safety management requirements

 The responsibilities of the Safety Manager are clearly defined along with identified lines of

communication within the organization

 Depending on the size and complexity of the airport’s operation, it may be useful to establish

a safety committee. The safety committee acts as a source of expertise for the Safety

Manager and is chaired by the Safety Manager

The written policy does outline specific responsibility and accountability. There is an airport safety

committee; however it only includes SCAS employees, no outside stakeholders. Additionally, safety

objectives should be set for the airport during the annual planning cycle with specific individuals

being responsible, and should be tied to their performance monitoring system. There is no safety

auditing system in place. SCAS has recently started tracking incident reports and applying

remediation to prevent future incidents but it is not formalized.

Assessment: Gapped

Requirements:

1. The existing policy does not adequately address the level of detail requisite in a fully capable

SMS program. As such, specific written guidance and an organization chart must be put into

place to clearly outline authority, responsibility and lines of communication associated with

SMS.

2. The airport safety committee should determine goals and objectives for its continued

operation in support of a complete SMS program and include outside stakeholders in

reporting, tracking and trending of hazards.

3. Safety objectives setting; implementing upon completion of the SMP

Overall Assessment Policy & Objectives: Emergent

Safety Risk Management6

SRM is a fundamental component of SMS. To be truly effective a SMS must have a formal risk

assessment program that identifies and documents hazards on the airport. An SMS:

 determines associated risk(s)

 identifies the severity and probability of the occurring risk(s)

6 AC 150/5200-37, page 9
Page 10 of 12

 develops mitigation strategies as appropriate

 applies, tracks, and monitors the mitigation strategy

 assesses and modifies strategies as necessary

There a partially organized SRM process is in place. It appears that safety programs existent at

SMF are based upon Human Resource and County practices of responding to regulation or incident.

It is the corporate commitment to SMS that will lead the organization towards a successful pilot

program as the first step followed by a fully encapsulating SMS program that will actually measure

hazards and implement steps to reduction. It will also need to embrace an auditing program to

ensure measures are successful.

Assessment: Gapped

Requirements: A fully implemented SRM process must be built from the ground up, which is

the primary focus of this pilot program effort. While SCAS has begun some reporting and tracking, it

is missing several key elements. The Safety Management Plan (SMP) will document a

comprehensive Hazard Assessment (HA), hazard mitigation development, risk modeling, and

assessment strategy as a deliverable from this pilot study process.

Safety Assurance7

Safety Assurance includes self-auditing, external auditing, and safety oversight. Safety oversight

can be achieved through auditing and surveillance practices, given the diverse activities at

commercial airports. In addition to the airport operator’s existing responsibilities for self-inspection

and correction of discrepancies under 14 CFR Part 139, an effective airport SMS audit program

should:

 Develop identified safety performance indicators and targets

 Monitor adherence to safety policy through self-auditing

 Allocate adequate resources for safety oversight

 Solicit input through a non-punitive safety reporting system

 Systematically review all available feedback from daily self-inspections, assessments,

reports, safety risk analysis, and safety audits

 Communicate findings to staff and implement agreed-upon mitigation strategies (14 CFR

Part 139 already requires this for actions covered by that regulation)

 Promote integration of a systems approach to safety into the overall operation of the airport

SMF has various safety programs associated with Part 139 requirements, as well as others which

include self-inspection and third-party inspection (OSHA, FAA, etc.). There is also the IIPP safety

program Countywide but there is no current program which promotes integration of various safety

7 AC 150/5299-37, page 12
Page 11 of 12

initiatives; there are no formal non-punitive reporting mechanisms; allocation of safety resources is

through HR and select staff on the Safety Committee. This needs to expand to be a collateral duty

by all employees and include a formal self-auditing process.

Assessment: Emergent

Requirement:

1. The SCAS will have to take the results of the SRM process and integrate it into a new Safety

Assurance program, to include:

a. Non-punitive reporting through use of a web-based tool (currently under development

by consultant)

b. Establish a means of collecting and trend analyzing disparate reports and data to

identify issues.

c. Develop and codify safety performance indicators.

d. Formalize their self auditing process,

e. Provide for resources for assurance functions (whether combined or separate to their

QMS assurance functions).

Safety Promotion8

Safety Promotion includes:

 Training and education

 Safety communication

 Safety competency and continuous improvement

The Safety Manager provides current information and training relating to safety issues relevant to

the specific operation of the airport. The provision of appropriate training to all staff, regardless of

their level in the organization, is an indication of management’s commitment to an effective SMS.

Safety training and education should consist of the following:

 A documented process to identify training requirements

 A validation process that measures the effectiveness of training

 Initial (general safety) job-specific training

 Recurrent safety training

 Indoctrination/initial training incorporating SMS

 Training that includes human factors and organizational factors

8 AC 150/5200-37, page 7
Page 12 of 12

There does not appear to be an integrated safety training system that would allow SCAS to monitor

and assess training effectiveness or compliance, outside the Part 139 requirement for driver training

for apron drivers, and the security training. Any new program should also integrate the various

stakeholder safety programs (FAA, Airline, others) in order to share lessons learned and best

management practices.

There does not appear to be a formal general safety communication. SCAS does have limited

personnel in place but promotion needs to be accepted organizationally wide as a philosophy not

just a Federal/OSHA, State or County requirement. SCAS’s commitment to continuous

improvement must also be formalized to support a robust SMS program.

Assessment: Emergent

Requirement: The following should be accomplished at the end of this pilot study.

1. Based upon the SRM process and the integration of various safety programs on the airport,

SCAS will document training requirements with measurable milestones for evaluation.

2. Training must be initiated for indoctrination training, recurrent training and job specific

(specialized) processes.

3. All training efforts will consider human factors in the process of reducing risk to as low as

possible (ALARP).

4. A robust safety communication means should be developed and managed, including lessons

learned assessments.

5. Safety Committee staff should have job specifications which outline their duties and be held

to performance standards, after proper training on safety and SMS has been provided to

ensure that staff understands what is expected from them.

6. SCAS needs to build and support a continuous improvement program .

Conclusion:

The Sacramento County Airport System has taken the first significant step to address each of the

areas listed as “GAP” in order to prepare one of the first fully operational SMS programs in the US.

None of the elements listed above face significant obstacles ahead in terms of drafting programs,

establishing processes or future implementation. The project continues to be on time and on budget.