# S MS Implementation St udy for

Jacksonville Aviation Authority (JAA)

COMPLETED BY:



### Wayne Clifton

, CPCU, CSP , PE

– A VP ESIS HSE Services

Virginia Griffin, CPCU, ALC M

, Sr. Consultant

Annapo l itan Business Suites

420 Chinquapin Round Rd

Suite I

- 2, Rm. 203

Annapolis, MD 21401

410

- 267

- 0185

Wayne.Clifton@esis.com

Virginia.Griffin@esis.com

DELIVERED TO :

Roger Studenski

Risk Management Administrator

14201 Pecan Park Road

Jacksonville FL 32218

904

- 741

- 3163

DATE OF PROPOSAL:



### January 7 , 201 1JAA SMS Implementation Strategy

January 7, 2011

Confidential January 7, 2011 Page 2 of 8 

## EXECUTIVE SUMMARY

Jacksonville Aviation Authority (JAA) and ESIS will develop, implement and report a

comprehensive and operable SMS Implementation Study as defined in the FAA Participant’s Guide,

Part 139 SMS Implementation Study of June 2010. The services will be performed for the Airport

under the Federal Aviation Admin istration (FAA) pilot program described in FAA Advisory Circular

AC 150/5200

- 37, ICAO Safety Management Manual Second edition, and ACRP Report 1, Safety

Management Systems for Airports Volumes 1 and 2 (O verview and G uidebook, respectively).

There are si x Study Tasks specifically requested by the FAA Guideline:

1. Implement Safety Risk Management procedures, processes, or policies as formulated under

the airport’s SMS Manual or other documentation developed for the airport under the first

pilot studies.

2. Con duct at least 3 safety risk analyses/assessments within 6 months of AIP grant award or study start. These analysis/assessments should not include analysis/assessments required

under FAA Air Traffic Organization SMS. The analysis/assessments can address haz ards in

the movement or non

- movement areas of the airport.

3. Implement a safety reporting and/or data collection system or applicable processes in

conformance with the airport’s SMS Manual or other documentation developed for the airport

under the first pilo t studies.

4. Collect hazard reports, incident and accident reports, and other safety related data/information

under the airport’s SMS Manual or other applicable documentation within 2 months of AIP

grant award or study start.

5. Analyze the information collected through the reporting and/or data collection system or

applicable processes within 5 months of AIP grant award or study start.

6. Conduct an internal audit/evaluation following the methods and procedures prescribed under

the Safety Assurance compone nt of the airport’s SMS ManualJAA SMS Implementation Strategy

January 7, 2011

Confidential January 7, 2011 Page 3 of 8 

## IMPLEMENTATION STRATE G Y

TASK 1

- Implement Safety Risk Management

Risk Management is the second component of the 4 SMS components , and includes, per the FAA

Circular, five phases:

Phase 1. Describe the system

Phase 2. Identify the hazards

Phase 3. Determine the risk

Phase 4. Assess and analyze the risk

Phase 5. Treat the risk (i.e., mitigate, monitor and track)

Additionally, FAA SMS concepts also expect a Risk Management process to :

appl y , track, and monitor the mitigation strategy and

assess and modif

y strategies as necessary

.

This is also described in the FAA Circular.

For this Task, the ESIS approach will be to:

Review the existing S RM procedures, processes and policies as developed under the first pilot

study . Determine if it fully meets and documents the SMS process.

E nsure roles and responsibilities are clearly defined, both in terms of input and outputs, indicators

and targets and accountabilities to ensure Risks are managed and abated and managed by the right

part of the organization

. This will involve not only reviewing the organizational relationships but

may include interviewing responsible parties to verify roles, responsibilities and accountabilities.

Ready these documents for the following Study Tasks.

Modify and adjust the inspection criteria (See Study Task 2)

In particular, the ESIS Risk Assessment Process is systematized and documented to ensure technical

accuracy, reproducibility and sustainability. The ESIS approach is consistent wi th the approach described

in the references. As illustrated below in Table 1 , our typical approach for risk assessment is a 6 step

process.JAA SMS Implementation Strategy

January 7, 2011

Confidential January 7, 2011 Page 4 of 8 Table 1. Risk Assessment Methodology

This approach and associated p lan will be completed and reviewed by JAA stakeholders before moving

on to Study Task 2.

Deliverables :

Written Procedure re

- drafted and edited to be fully inclusive of the process, roles and responsibilities

and assessment methodologies.

Risk Assessment Methodology (and Excel table)

Revis ed inspection criteria methodology process (See Task 2)

.



### TASK 2

- Conduct at least 3 safety risk analyses/assessments

For this Task, t he ESIS Team will work with JAA to identify three areas or tasks for risk assessment.

Typically these are performed in higher risk areas, one each in the movement and non

- movement

areas, the other can be a collective involvement of multiple stakeholders. The first risk assessment

will involve hazards associated with “Wildlife”. We will expect to involve numerous stakeho lders in

each risk assessment itself; preceding the assessment with some training in the process while

soliciting stakeholder input into hazard and control identification, severity and likelihood

determination. The ESIS Team will lead and facilitate these assessments face to face, the first one to

coincide with the first of the four required quarterly s ite visits.JAA SMS Implementation Strategy

January 7, 2011

Confidential January 7, 2011 Page 5 of 8 The typical outline of each 6

- 8 hour risk assessment effort is :

A two hour int r o duction to the risk assessment process with the stakeholder group

Preliminary risk assessment to include control selection

Risk and mitigation strategy determination.

After mitigation strategies have been implemented; one of the ESIS team members will perform a

field walkthrough and verification of hazard and controls and finalize the risk assessment document.

Deliverables :

Two face

- to

- face risk assessment meetings (one to coincide with the quarterly visit. The other

two will be scheduled back to back to minimize travel expenses)

Three risk assessments (in E xcel)

. Risk assessment training PowerPoint.



### TASK 3

- Implement a safety reporting and/or data collection sy stem

ESIS will evaluate and make recommendations on the JAA SMS hazard reporting system to verify that it

meets the SMS Manual and FAA SMS expectations.

There are two parts to Study Task 3. One is the safety (hazard) reporting, the other is the data collection.

From a safety hazard reporting standpoint, t he SMS should include a visible non

- punitive safety reporting

system supported by management. The safety reporting system shou ld permit feedback from personnel

regarding hazards and safety

- related concerns. The SMS should use this information to identify and

address safety deficiencies. The safety reporting system may also identify and correct non

- conformance to

safety policy. T his includes not only safety reporting of personnel concerns, but safety reporting as

identified by existing inspection and other systems. Some of these inspection s may include the Part

139 FAA required inspections as well as other inspections performed b y airport operations and their

tenants.

Finally, the ability to collect and analyze this data is imperative to a functioning SMS, along with

developing indicators and targets for these data points.

The ESIS Team approach will be to inventory the exist ing data collection media and collection

systems, both on the part of airport and tenant operations and SMS expectations. Our evaluation will

help JAA to develop both incentives and management accountably for the encouragement of concern

reporting, and he lp JAA develop and implement incentives and management metrics and

accountabilities so that reporting is encouraged and rewarded. In addition, SMS expects the data to

not only be collected, but trended and analyzed (Study Step 5).

Deliverables :

Lists of data collection media

Categories of safety concerns

Develop indicator and targets for the organization to encourage reporting

Incentive or recognition approaches that organizational management can use to encourage

data collection.JAA SMS Implementation Strategy

January 7, 2011

Confidential January 7, 2011 Page 6 of 8 

### TASK 4

- Collect hazard reports, incident and accident reports, and other safety

related data/information

The ESIS Team will conduct an analysis of the JAA SMS hazard reporting system and making

recommendations for improvements. It is our expectation that there are numerous exi sting

inspections that occur within the airport. Some of these are required by regulation (e.g., Part 139,

OSHA) . O thers

, such as preventative maintenance activities

, may also identify hazards associated

with critical control equipment (alarms, sensors, v ehicles, etc

. ). Thus, there are a number of

organizations within the airport that help identify hazards, i.e., fire dept, security, etc. These hazard

identification sources will be inventoried and collected.

From an incident and accident report stand point, again, a number of organizations perform these

activities. These sources will be inventoried and collected.

Deliverables :

List of Hazard Reports / Sources

List of Incident and accident reports / Sources

Lists of improvement recommendations



### TASK 5

- Analyze the information collected

The ESIS Team approach w ill be to demonstrate and document an analytical process that sorts and

trends the:

inspection data,

reporting data

conformance to high risk controls

hazards identified

closure rate on correct ive actions and mitigations and

achievement of indicators and targets.

The ESIS Team will review, collate and analyze the first three months of data and present it in a

systematic scorecard type approach many of our clients have found beneficial. This approach will be

documented so JAA can reproduce it in a sustainable manner.

Deliverable s :

A process for sorting and analyzing data and

A scorecard of indicators and targets utilizing this data.JAA SMS Implementation Strategy

January 7, 2011

Confidential January 7, 2011 Page 7 of 8 

### TASK 6

- Conduct an internal audit/evaluation (Assessment)

Conduct an internal audit/evaluation ( Gap Analysis ) of the Safety Risk Management related

elements (for example, the ISAGO criteria for Risk Assessment and Safety Assurance), following the

methods and procedures prescribed under the Safety As surance component of the airport’s SMS

Manual and the other FAA references

.

In order to perform this analysis , we use a combination of three methods. Specifically:

Documentation (Record) review

Interviews with personnel, both Airport and cognizant stakeh olders that impact these SMS

elements

. Physical survey of the three risk

- assessed area s

Each of the above helps validate and verify the findings and conclusions of the other.

The physical survey portion is then performed to verify how your existing processes may meet SMS

Risk Management expectations. Various records and action plans are tested. Some additional topic

specific interviews may be performed at this time. Again, this survey will coin cide with one of the

four required on site quarterly meetings.

Scoring Process

ESIS uses a qualitative and quantitative gap analysis /audit process. Th is component consist s of a

number of elements, for which we will also evaluate, score and describe any findings. The scoring

process is consistent with a methodology used for over 20 years, which helps benchmark site

programs and efforts against equivalent level programs. Thus, for each criterion, there are five

possible answers and points awarded:

Does Not Meet 0 points = Not found

Under Development 1 point = Beginning, documented but not implemented

Partially Meets 2 points = implemented but missing in either documentation,

consistency, thoroughnes s or effectiveness

Meets 3 points, and

Not Applicable null

Findings are recorded, as are Responses (R), Points Received (PR), and Points Available (PA). The

points received are divided by the points available to yield a percentage score, based on a 0

- 100 scale.

Within each Element, points are totaled, and compared to the total points available, yielding a score

from 0

- 100%. Based on over 300 surveys performed over the last 18 years, the scoring methodology

is:

Table 2 Scoring Methodology

0

- 50% Gaps exist in the program, process, or effectiveness

51

- 79% The program or process has some vulnerabilities in providing consistent, sustainable results

80

- 100% While there still may be some recommendations for improvement, the overall program or

process is effective in design and implementation.JAA SMS Implementation Strategy

January 7, 2011

Confidential January 7, 2011 Page 8 of 8 Our customers have found that by combining a qualitative and quantitative gap analysis, relative

strengths and weaknesses are not only better determined, it creates a baseline against which true

continuous improvement can be measured. This information is extremely valuable as a baseline for

future assessment so true SMS progress can be evaluated and verified.

Deliverables

Gap Analysis Scor e

Gap Analysis Narrative Report