SMS Implementation Study for

Jacksonville Aviation Authority (JAA)

COMPLETED BY:

Wayne Clifton, CPCU, CSP, PE – AVP ESIS HSE Services

Virginia Griffin, CPCU, ALCM, Sr. Consultant

Annapolitan Business Suites

420 Chinquapin Round Rd

Suite I-2, Rm. 203

Annapolis, MD 21401

410-267-0185

Wayne.Clifton@esis.com

Virginia.Griffin@esis.com

DELIVERED TO:

Roger Studenski

Risk Management Administrator

14201 Pecan Park Road

Jacksonville FL 32218

904-741-3163

DATE OF PROPOSAL:

January 7, 2011
JAA SMS Implementation Strategy

January 7, 2011

Confidential January 7, 2011 Page 2 of 8

EXECUTIVE SUMMARY

Jacksonville Aviation Authority (JAA) and ESIS will develop, implement and report a 

comprehensive and operable SMS Implementation Study as defined in the FAA Participant’s Guide, 

Part 139 SMS Implementation Study of June 2010. The services will be performed for the Airport 

under the Federal Aviation Administration (FAA) pilot program described in FAA Advisory Circular 

AC 150/5200-37, ICAO Safety Management Manual Second edition, and ACRP Report 1, Safety 

Management Systems for Airports Volumes 1 and 2 (Overview and Guidebook, respectively). 

There are six Study Tasks specifically requested by the FAA Guideline:

1. Implement Safety Risk Management procedures, processes, or policies as formulated under 

the airport’s SMS Manual or other documentation developed for the airport under the first 

pilot studies.

2. Conduct at least 3 safety risk analyses/assessments within 6 months of AIP grant award or 

study start. These analysis/assessments should not include analysis/assessments required 

under FAA Air Traffic Organization SMS. The analysis/assessments can address hazards in 

the movement or non-movement areas of the airport.

3. Implement a safety reporting and/or data collection system or applicable processes in

conformance with the airport’s SMS Manual or other documentation developed for the airport 

under the first pilot studies.

4. Collect hazard reports, incident and accident reports, and other safety related data/information 

under the airport’s SMS Manual or other applicable documentation within 2 months of AIP 

grant award or study start.

5. Analyze the information collected through the reporting and/or data collection system or

applicable processes within 5 months of AIP grant award or study start.

6. Conduct an internal audit/evaluation following the methods and procedures prescribed under 

the Safety Assurance component of the airport’s SMS Manual
JAA SMS Implementation Strategy

January 7, 2011

Confidential January 7, 2011 Page 3 of 8

IMPLEMENTATION STRATEGY

TASK 1- Implement Safety Risk Management

Risk Management is the second component of the 4 SMS components, and includes, per the FAA 

Circular, five phases:

Phase 1. Describe the system 

Phase 2. Identify the hazards 

Phase 3. Determine the risk 

Phase 4. Assess and analyze the risk 

Phase 5. Treat the risk (i.e., mitigate, monitor and track) 

Additionally, FAA SMS concepts also expect a Risk Management process to:

apply, track, and monitor the mitigation strategy and 

assess and modify strategies as necessary.

This is also described in the FAA Circular.

For this Task, the ESIS approach will be to: 

Review the existing SRM procedures, processes and policies as developed under the first pilot 

study. Determine if it fully meets and documents the SMS process.

Ensure roles and responsibilities are clearly defined, both in terms of input and outputs, indicators 

and targets and accountabilities to ensure Risks are managed and abated and managed by the right 

part of the organization. This will involve not only reviewing the organizational relationships but 

may include interviewing responsible parties to verify roles, responsibilities and accountabilities.

Ready these documents for the following Study Tasks.

Modify and adjust the inspection criteria (See Study Task 2)

In particular, the ESIS Risk Assessment Process is systematized and documented to ensure technical 

accuracy, reproducibility and sustainability. The ESIS approach is consistent with the approach described 

in the references. As illustrated below in Table 1, our typical approach for risk assessment is a 6 step 

process.
JAA SMS Implementation Strategy

January 7, 2011

Confidential January 7, 2011 Page 4 of 8

Table 1. Risk Assessment Methodology

Step 1: Identify your Tasks Step 2: Identify your Risks, Threats and Hazards

Step 3: 

Identify your 

Controls: Refer to Part 139 if Applicable

Less Effective Controls

Preferred Controls Department Task Hazard Category (What could go wrong) Hazard Aspect (Types) Scenario (outcome) Severity Description Part 139 Task Applicability Controls: Elimination/Substitution Controls: Engineering Controls: Administrative/warning Controls: Personal Protective Equipment (PPE) Description of Controls

Step 4: Determine Risk 

Factors and Evaluate 

Risk

Step 5: Corrective Action and 

Implementation Step 6 Risk Management and Risk Reduction Residual Severity Residual Probability Residual Risk Total Corrective Actions Corrective Action (CA) Assigned to (by Department) Critical Control (Y if Residual Severity is Catastrophic or Serious, or if Residual Risk is High) Describe the Critical Control? (add to inspection, testing or observation) Critical Control Owner (by Title) Control Category (based on Corrective Action) Hierarchy/Defense in Depth Met (If No, then add additional controls via CAs is Step 5) Risk Reduction Target 

This approach and associated plan will be completed and reviewed by JAA stakeholders before moving 

on to Study Task 2.

Deliverables:

Written Procedure re-drafted and edited to be fully inclusive of the process, roles and responsibilities 

and assessment methodologies.

Risk Assessment Methodology (and Excel table) 

Revised inspection criteria methodology process (See Task 2).

 

TASK 2- Conduct at least 3 safety risk analyses/assessments

For this Task, the ESIS Team will work with JAA to identify three areas or tasks for risk assessment. 

Typically these are performed in higher risk areas, one each in the movement and non-movement 

areas, the other can be a collective involvement of multiple stakeholders. The first risk assessment 

will involve hazards associated with “Wildlife”. We will expect to involve numerous stakeholders in 

each risk assessment itself; preceding the assessment with some training in the process while 

soliciting stakeholder input into hazard and control identification, severity and likelihood 

determination. The ESIS Team will lead and facilitate these assessments face to face, the first one to 

coincide with the first of the four required quarterly site visits.
JAA SMS Implementation Strategy

January 7, 2011

Confidential January 7, 2011 Page 5 of 8

The typical outline of each 6-8 hour risk assessment effort is:

A two hour introduction to the risk assessment process with the stakeholder group

Preliminary risk assessment to include control selection

Risk and mitigation strategy determination.

After mitigation strategies have been implemented; one of the ESIS team members will perform a 

field walkthrough and verification of hazard and controls and finalize the risk assessment document.

Deliverables:

Two face-to-face risk assessment meetings (one to coincide with the quarterly visit. The other 

two will be scheduled back to back to minimize travel expenses)

Three risk assessments (in Excel). 

Risk assessment training PowerPoint.

TASK 3- Implement a safety reporting and/or data collection system

ESIS will evaluate and make recommendations on the JAA SMS hazard reporting system to verify that it 

meets the SMS Manual and FAA SMS expectations. 

There are two parts to Study Task 3. One is the safety (hazard) reporting, the other is the data collection. 

From a safety hazard reporting standpoint, the SMS should include a visible non-punitive safety reporting 

system supported by management. The safety reporting system should permit feedback from personnel 

regarding hazards and safety-related concerns. The SMS should use this information to identify and 

address safety deficiencies. The safety reporting system may also identify and correct non-conformance to 

safety policy. This includes not only safety reporting of personnel concerns, but safety reporting as 

identified by existing inspection and other systems. Some of these inspections may include the Part 

139 FAA required inspections as well as other inspections performed by airport operations and their 

tenants. 

Finally, the ability to collect and analyze this data is imperative to a functioning SMS, along with 

developing indicators and targets for these data points. 

The ESIS Team approach will be to inventory the existing data collection media and collection 

systems, both on the part of airport and tenant operations and SMS expectations. Our evaluation will 

help JAA to develop both incentives and management accountably for the encouragement of concern 

reporting, and help JAA develop and implement incentives and management metrics and 

accountabilities so that reporting is encouraged and rewarded. In addition, SMS expects the data to 

not only be collected, but trended and analyzed (Study Step 5).

Deliverables:

Lists of data collection media

Categories of safety concerns

Develop indicator and targets for the organization to encourage reporting

Incentive or recognition approaches that organizational management can use to encourage 

data collection. 
JAA SMS Implementation Strategy

January 7, 2011

Confidential January 7, 2011 Page 6 of 8

TASK 4-Collect hazard reports, incident and accident reports, and other safety 

related data/information

The ESIS Team will conduct an analysis of the JAA SMS hazard reporting system and making 

recommendations for improvements. It is our expectation that there are numerous existing 

inspections that occur within the airport. Some of these are required by regulation (e.g., Part 139, 

OSHA). Others, such as preventative maintenance activities, may also identify hazards associated 

with critical control equipment (alarms, sensors, vehicles, etc.). Thus, there are a number of 

organizations within the airport that help identify hazards, i.e., fire dept, security, etc. These hazard 

identification sources will be inventoried and collected. 

From an incident and accident report standpoint, again, a number of organizations perform these 

activities. These sources will be inventoried and collected. 

Deliverables:

List of Hazard Reports / Sources

List of Incident and accident reports / Sources

Lists of improvement recommendations

TASK 5- Analyze the information collected

The ESIS Team approach will be to demonstrate and document an analytical process that sorts and 

trends the:

inspection data, 

reporting data

conformance to high risk controls

hazards identified

closure rate on corrective actions and mitigations and 

achievement of indicators and targets.

The ESIS Team will review, collate and analyze the first three months of data and present it in a 

systematic scorecard type approach many of our clients have found beneficial. This approach will be 

documented so JAA can reproduce it in a sustainable manner. 

Deliverables:

A process for sorting and analyzing data and 

A scorecard of indicators and targets utilizing this data.
JAA SMS Implementation Strategy

January 7, 2011

Confidential January 7, 2011 Page 7 of 8

TASK 6- Conduct an internal audit/evaluation (Assessment)

Conduct an internal audit/evaluation (Gap Analysis) of the Safety Risk Management related 

elements (for example, the ISAGO criteria for Risk Assessment and Safety Assurance), following the 

methods and procedures prescribed under the Safety Assurance component of the airport’s SMS 

Manual and the other FAA references.

In order to perform this analysis, we use a combination of three methods. Specifically:

Documentation (Record) review

Interviews with personnel, both Airport and cognizant stakeholders that impact these SMS 

elements.

Physical survey of the three risk-assessed areas

Each of the above helps validate and verify the findings and conclusions of the other.

The physical survey portion is then performed to verify how your existing processes may meet SMS 

Risk Management expectations. Various records and action plans are tested. Some additional topicspecific interviews may be performed at this time. Again, this survey will coincide with one of the 

four required on site quarterly meetings.

Scoring Process

ESIS uses a qualitative and quantitative gap analysis/audit process. This component consists of a 

number of elements, for which we will also evaluate, score and describe any findings. The scoring 

process is consistent with a methodology used for over 20 years, which helps benchmark site 

programs and efforts against equivalent level programs. Thus, for each criterion, there are five 

possible answers and points awarded:

Does Not Meet 0 points = Not found

Under Development 1 point = Beginning, documented but not implemented

Partially Meets 2 points = implemented but missing in either documentation, 

consistency, thoroughness or effectiveness

Meets 3 points, and 

Not Applicable null

Findings are recorded, as are Responses (R), Points Received (PR), and Points Available (PA). The 

points received are divided by the points available to yield a percentage score, based on a 0-100 scale.

Within each Element, points are totaled, and compared to the total points available, yielding a score 

from 0-100%. Based on over 300 surveys performed over the last 18 years, the scoring methodology 

is:

Table 2 Scoring Methodology

0-50% Gaps exist in the program, process, or effectiveness

51-79% The program or process has some vulnerabilities in providing consistent, sustainable results

80-100% While there still may be some recommendations for improvement, the overall program or 

process is effective in design and implementation.
JAA SMS Implementation Strategy

January 7, 2011

Confidential January 7, 2011 Page 8 of 8

Our customers have found that by combining a qualitative and quantitative gap analysis, relative 

strengths and weaknesses are not only better determined, it creates a baseline against which true 

continuous improvement can be measured. This information is extremely valuable as a baseline for 

future assessment so true SMS progress can be evaluated and verified.

Deliverables

Gap Analysis Score

Gap Analysis Narrative Report