# Presentation: 104630

## Data Quality and Compliance

## Data Quality and Compliance

- In order for a GDP to produce the desired results, the program must be based on accurate data and flights must arrive at their designated CTAs.
- Data Quality
- An accurate prediction of arrival demand is critical to planning and managing a ground delay program. Poor data quality results in inaccurate predictions.
- Compliance to Control Times
- Non-compliance results in the program delivering above or below the airport acceptance rate.

## What do we mean by GDP Compliance?

- EDCT Compliance
- Did flights depart within  (-5, +5) minutes of their EDCTs?
- ETE
- Do airline filed ETEs (PETE and LRTA-LRTD) differ significantly from actual ETEs?
- CTA Compliance
- Did flights arrive within (-5, +5) minutes of their CTAs?

## Factors that Impact Performance

- Compliance To Control Times
- Data Quality
- Pop-ups
- Time Out Delays
- Processing Errors
- Forecast Accuracy
- En route Times
- Arrival Time Estimates
- Slot Usage
- Operational Practices

## Impact of Poor Data Quality and Non-Compliance

- Unnecessary ground delay programs
- Flights may take longer delays than necessary.
- Program may deliver light or heavy
- Unnecessary airborne holding
- More revisions
- GDP may be longer than necessary
- “Rolling Spike”

## How to fix the problems:Real-Time Monitoring

- Motivation: Provide traffic management specialists with a tool that will allow them to spot problems and take corrective action while the program is executing.
- FSM
- Real-Time Flight Schedule Analyzer (RT FSA)
- ETMS lists

## FSM Data Quality & Compliance Monitoring Capabilities

- Flight Schedule Monitor (FSM)
- Alerts
- CTA and EDCT compliance
- ETE Verses Actual
- Reports
- Timeout cancellations
- Timeout delays
- Flight lists - duplicate flights, schedule verification
- Color by Alarm
- Color by compliance

**Notes:**
Duplicate flights - flights filed under 2 different call signs for sub-carriers or code shares

Schedule Verification -  checking that the flights in FSM match what the airline is planning on operating so they can cancel ones that aren’t going to fly.

## Real-Time Flight Schedule Analyzer

- Real-Time FSA (RT FSA) is a web-based application that allows monitoring of ground delay programs (GDPs) as they are executing.  FSA generates nine different types of reports:

- The reports are updated every five minutes with the most current Airport Demand List (ADL) information.

- Time-Out Delay
- Pop-Ups
- Duplicate Flights
- CTD before PGTD
- Compliance

- Cancels That Flew
- GDP Events
- Flight Status
- Performance

- Available to all Users:
- ATCSCC
- ARTCC

- TRACON
- Airlines/CDM participants (Filtered)

## Real-Time FSA: Accessing the Reports

- Real-Time FSA: Accessing the Reports

- The reports are viewed using Netscape.  To access the reports:
- 1.	Start Netscape
- Go to the ATCSCC homepage: http://www.atcscc.tfmi.nas.faa.gov/
- 3.	Select the ‘Operational Information System (OIS)’ link.
- 4.	Select the FSA link on the bottom of the left hand side menu on the OIS page.

- 3.

- 4.

## Real-Time FSA Reports:Some Capabilities

- Real-Time FSA Reports:Some Capabilities

- Identify flights needing updated airline departure/arrival times (Time Out delayed)

- Identify unscheduled (pop-up) flights causing over-delivery

- Check EDCT Compliance trends

**Notes:**
Performance reports show whether the GDP is delivering the requested rate and indicate why the desired rate is not being achieved.  Compliance Reports allows FAA personnel to monitor compliance to assigned control times.  Additional reports help identify data quality problems that may negatively impact program performance. 

## Real-Time FSA: Flight Performance Analysis 

- Click on a link to display the list of flights included in the counts. The “Comments” column in the flight list provides an indication of why a flight either does not have a slot or is not arriving at its assigned slot time.
- A running history of individual flights can be accessed for detailed troubleshooting.

## Real-Time Capabilities to Make a Positive Impact on Performance

- Airline
- Simplified Substitutions
- Duplicate flight records
- Flight operations updates
- Delays
- Cancellations
- Scheduling additional flights
- ETE estimates

- ATCSCC
- EDCT Updates with ECR
- Ground Delay Program Adjustments with FSM
- Revisions/Extensions
- Compressions
- EDCT suspensions

## How to prevent future problems: Next-Day Analysis

- Motivation: What can we learn from past programs that will help us improve future programs?
- Tools Available to all CDM participants:
- ATCSCC position logs
- Morning Brief Reports
- Tools currently provided to the ATCSCC and field facilities:
- Post Analysis FSA
- EDCT log
- EDCT Compliance lists (w/GDP Summary)

## ATCSCC Position Logs

- Access through the ATCSCC Intranet: http://www.atcscc.faa.gov/

## Morning Brief Reports

- Access through QA page on the ATCSCC Intranet: http://www.atcscc.faa.gov/

## Morning Brief: GDP Delivery

## Morning Brief: Departure Compliance Distribution

## Morning Brief: Departure Compliance sorted by...

## Morning Brief: Compliance Correlation

## Morning Brief: Pop-Ups

## Morning Brief: Unused Slots

## Post Analysis FSA (ATCSCC QA department)

- Client/Server application designed to
- Measure GDP performance
- Identify factors that impact performance
- Monitor known factors
- Measures GDP performance by looking at
- Aircraft landed vs.. airport AAR/program rate
- Delay (ATC, Absolute)
- Airborne Holding
- Rate Control Index (RCI)
- Used to generate the Morning Brief Report

## Post Analysis FSA (cont.)

- Analysis separated into 7 sections:
- GDP Summary
- GDP Delivery
- Delay
- Compliance
- Time Out Delays
- Pop-Ups
- Cancellations

## EDCT Compliance Lists (FAA field facilities)

- Automated List from FSA Database
- Expandable detail list on each flight

## EDCT Log (FAA field facilities)

- Parsing of ATCSCC Logs included in EDCT Compliance Email

## Accomplishments

- FSA has enabled ATCSCC staff to
- Monitor compliance to control times
- Identify data processing errors and other data quality problems
- Identify operational practices that lead to under or over delivery
- Identify issues with current slot assignment procedures

**Notes:**
   Examples of data processing errors identified using FSA:
	Slot times that disappear in the ADLs
	FA delays assigned to scheduled flights
	CTAs < slot time by as much as 4 hours
	ETA<ETD

Examples of system manipulation identified using FSA:
	Filing longer en-route times to obtain earlier CTDs
	Filing extra flight to get additional slots for subbing

Examples of slot assignment algorithm issues due to policy choices identified using FSA
CTA <CTD for non-departed flights

## Back-Up Slides

## Data Quality

- What do we mean by data quality?
- Accurate Information
- Known in Advance
- Improve information for decision making
- Early detection of problem allows earlier correction to program
- Quicker attention to non-compliance will get the information from those involved
- Less resources at field faculties required for follow up investigation.

## Airline Data Quality Impacts

- Cancellations
- Were cancellation (SI or FX) messages sent to for cancelled flights?
- Delay Information
- Were flight modify (FM) messages sent for delayed flights?
- New Flights
- Were flight create (FC) messages sent for new flights?
- ETE Estimation
- Does the actual time en route differ significantly from the airline filed ETE?
- Duplicate Flight Records
- Did the sub and parent carriers submit multiple flight records for the same flight?

## Post Analysis FSA: GDP Summary

## Post Analysis FSA: GDP Delivery

**Notes:**
Performance  Metrics
Aircraft landed vs. airport AAR/program rate
Rate Control Index (RCI)
Delay (ATC, Absolute)
Airborne Holding

Factors That Impact Performance
Compliance To Control Times
Data Quality (Pop-ups, Timeouts, Processing Errors )
Forecast Accuracy (En route Times, Arrival Times)
Slot Usage
Operational Practices

## Post Analysis FSA: Delay

## Post Analysis FSA: Compliance

## Post Analysis FSA: Time Out Delayed Flights

## Post Analysis FSA: Unscheduled (Pop-Up) Flights

## Post Analysis FSA: Pop-Up Trend Analysis

## Post Analysis FSA: Cancellations and Slot Usage