MEMORANDUM

TO: Bryan Manning, EPA Office of Transportation and Air Quality ( OTAQ )

FROM: Richard Billings E astern Research Group, Inc. (E RG )

DATE: April 6 , 2015

SUBJECT: Documentation of Quality Checks Implemented and Peer Review Process for Small Jet Fuel Use and Carbon Dioxide ( CO 2 ) Emissions Inventory

1.0 Introduction

ERG develop ed an emission inventory of aircraft and helicopters equipped with engines that burn jet fuel that will not be required to comply with carbon dioxide standards being developed by I nternational C ivil A viation O rganization ’s (ICAO) Committee on Aviation Environmental Protection (CAEP) . A preliminary dataset and report was completed in December 2014 and provided to the EPA for review.

In addition to EPA’s and ERG’s internal review, two independent consultants were commissioned to conduct an external review to assess the methodology, d ata sources, and results of the preliminary data.

This memorandum summarizes the internal quality checks that were implemented along with documentation of the peer review process. The external reviewers’ comments and observations are documented in a sepa rate report that compiles all comments and revisions made to the report. In addition to this memorandum, the peer review record includes:

• Jet Fuel Use and CO 2 Emissions Inventory for Aircraft below ICAO GHG Program Thresholds

- D raft Report with compiled external review comments

.

• Jet Fuel Use and CO 2 Emissions Inventory for Aircraft b elow ICAO GHG Program Thresholds

- Final Report

.

2.0 Internal Quality Checks

To ensure the quality of data used in the study and reasonableness of the result s, the following quality assurance checks were implemented:

1• Checked data compi lation to ensure that the datas ets matched the original data set and were not corrupted during data transfer. No issues with data transfer or transcription were found.

• Compared landing and takeoff ( LTO ) fuel usage rates to aircraft average data to identify and evaluate aircraft that were significantly different for the average values . LTO fuel usage was only used for the T

- 100 data that were applied to Emissions and Dispersion Mo deling System ( EDMS ) , which includes separate fuel usage data for aircraft make/model engine make/model combinations. No alternative LTO fuel usage data sources were identified or used in this study.

• Compared CO 2 fuel fractions found in EDMS (9.75 kg CO 2 /g al)

, the 2008 Climate Leader s GHG Guidance (9.57 kg CO 2 /gal) (EPA, 2008), the 2014 EPA Fast Facts: National

- Level U.S. Greenhouse Gas Inventory summaries (EPA, 2014b)

, and the 2015 draft Inventory of U.S. GHG Emissions and Sinks re port (9.75 kg/CO 2 ) (EPA, 2015)

. The EPA recommended using the 9.75 kg/CO 2 because it is based on the most recent data and is consistent with the EDMS factor.

• E nsured that the LTO data used in the EDMS runs matched the data compiled from the T

- 100 data.

• Compared fuel usage rates f or cruising to Project Interactive Analysis and Optimi z ation ( PIANO ) data provided by the EPA. The PIANO fuel usage rate data were applicable to only aircraft above the maximum takeoff mass ( MTOM ) threshold and were not used directly in this assessment of aircraft below the MTOM threshold.

• A ssessed the fraction of CO 2 emissions associated with the aircraft relative to total jet CO 2 emissions in the 2015 draft Inventory of U.S. GHG Emissions and Sin ks (2015) and Fast Facts: National

- Level U.S. Greenhouse Gas Inventory (EPA, 2014 ).

• Compared the aircraft counts from the General Aviation and Air Taxi Activit y ( GAATA ) survey, the FAA Registry, and the Ascend database.

Table 2

- 1 shows general Quality Assu rance assessment factors and how they were considered in this study

.

2Table 2

- 1

. QA Summary Table

General Assessment Factor Definition How the Factor was Considered Soundness The extent to which the scientific and technical procedures, measures, methods or models employed to generate the information are reasonable for, and consistent with, the intended application. ERG used accepted inventory procedures to compile available da ta and calculate emissions. The emission calculations follow similar methods used in the EPA’s 2011 National Emission Inventory. Applicability and Utility The extent to which the information is relevant for the Agency’s intended

use. There is no singular data source currently available to quantify aircraft counts and activity levels for aircraft with MTOM below the CO 2 standard thresholds

. To get the required data, T

- 100 datasets, which includes aircraft make and model

- specific information were used in conjunction with the FAA EDMS model to identify commercial aircraft below the thresholds

. GAATA data along with aircraft registration data were used to quantify non

- commercial aircraft (though adjustments were needed to account for air taxis which would appear in both studies

. Clarity and Completeness The degree of clarity and completeness with which the data, assumptions, methods, quality assurance, sponsoring organizations

, and analyses employed to generate the information are documented. Throughout the process, data were provided to the EPA to show how the datasets were being developed, including summary comparisons and analysis.

The quality checks implemented in this study were included in the project Quality Assurance Project Plan (QAPP )

. This document, in conjunction with the project datasets

, was developed to allow for independent replication of the procedures used.

3Table 2

- 1

. QA Summary Table

General Assessment Factor Definition How the Factor was Considered Uncertainty and Variability The extent to which the variability and uncertainty (quantitative and qualitative) in the information or in the procedures, measures, methods or models are evaluated and characterized. The data used in this study were either detailed and specific ( such as the T

- 100 and EDMS factors ) or highly aggregated ( such as the GAATA activity data ) ; making it difficult to quantitatively assess uncertainty.

To the extent possible, information is provided in the report to qualitatively indicate whether the data over

- or under

estimated actual fuel usage or emissions. Evaluation and Review The extent of independent verification, validation and peer review of the information or of the procedures, measures, methods

, or models. Internally, data were reviewed at critical points of the project to insure that data were compiled and calculations were implemented correctly.

The EPA implemented a comprehensive review of the report and data; all recommended changes are included in memo

.

Glenn McDonald and Peter Zimm from ICF and an independent contractor

, Ralph Rissmiller of RR Aviation Services, LLC

, provided an external review of the methodology used and preliminary results. Suggested changes were addressed in the current version of this report.

External Peer Review

The peer review process began in December 201 4 and was completed in February 201 5

. The EPA Work Assignment M anager identified t hree external independent reviewers who have experience with aviation emission inventory studies and who have active support contracts with

4OTAQ . These reviewers were provided with a copy of the draft report and associated data

. The peer reviewers were asked to draw upon their expertise in small aircraft, as well as in GHG emission estimation methods, to comment on whether the data inputs, approach, and methodologies presented in the draft report reflect sound scientific and analytical practice, and adequately quantified the fraction of CO 2 emissions associated with aircraft below the ICAO GHG reporting thresholds. Table 3

- 1 identifies the peer reviewers participating in the review and their organizational affil iations. No changes were recommended for the data developed in this study.

Table 3

- 1

. External Independent Peer Reviewers

Reviewer Affiliation Glenn Mc Donald Peter Zimm ICF

Ralph R issmiller RR Aviation Services, LLC

The peer reviewers provided written responses to the EPA Work Assignment Manager who provided them to ERG for consideration in revising the approach or documentation. These comments were summarized in a spreadsheet that indicated the reviewer, page/line

nu mber, draft text for which the comment is associated, the question/issue , and response

. Table 3

- 2 of this memorandum summarizes these comments and responses

. Note, the peer reviewers did not provide any recommendations or suggestions that required changes to the project dataset.

5Table 2. Comment/Response to the Draft

Commenter Page Section Text Comment Response Glenn McDonald/ Peter Zimm 1.0 General Comment It might be helpful to outline the overall methodology visually in a diagram in Section 1.0 Chart added

Glenn McDonald/ Peter Zimm General Comment Possibly an issue with word docs being converted to PDF, but there are a few issues in font standardization and word spacing Checked PDF files for issue Glenn McDonald/ Peter Zimm General Comment Define acronyms first time using (GAATA, GHG, QA/QC, etc

. ) Checked acronyms to ensure they

are all defined when first mentioned in the text Glenn McDonald/ Peter Zimm 3 General Comment Might be helpful to state that LTO cycles and cruising time are on annual basis Annual time notation included in the
text

Ralph Rissmiller 5 1.0 In addition, CAEP’s applicability thresholds exclude piston

- engine aircraft and helicopters, and since CAEP requirements only apply to civil aircraft, the aircraft CO 2 standard also does not apply to military aircraft. The footnote indicates that piston

engine aircraft operate on aviation gasoline, however a growing number may be operated on Jet

- A fuel (i.e., diesel powered piston

engine aircraft). Footnote updated with this information

Ralph Rissmiller 5 1.0 In addition, CAEP’s applicability thresholds exclude piston

- engine aircraft and helicopters, and since CAEP requirements only apply to civil aircraft, the aircraft CO 2 standard also does not apply to military aircraft. Helicopters are powered by both aviation gasoline and Jet

- A fuel. For applicability purposes both fuel types would still be excluded. No change required.

Ralph Rissmiller 5 1.0 In addition, CAEP’s applicability thresholds exclude piston

- engine aircraft and helicopters, and since CAEP requirements only apply to civil aircraft, the aircraft CO 2 standard also does not apply to In my opinion, these aircraft generate a proportionally larger amount of CO 2 emissions due to their generally lower fuel efficiency than a comparable Good po int, but this is a discussion of applicability and military aircraft are not included in the standard. No change required.

6Table 2. Comment/Response to the Draft

Commenter Page Section Text Comment Response military aircraft. commercial aircraft.

Ralph Rissmiller 6 1.0 To ensure the products for this project align with data used in the Emission and Sinks document, data obtained from the Federal Aviation Administration’s (FAA) 2012 General Aviation Part 135 Activity Surveys (GAATA, 2013) was assumed to be fuel purchased and used within the U.S. I believe this is generally correct, however a significant number of business jets and turboprops are operated under Part 91 (not 135). There are also specific organizations that operated some of their flights under Part 91 and some under Part 135 criteria. For example, NetJets (a large fractional operator) operates under Part 91 Su bpart K. Clarifying text added

Glenn McDonald/ Peter Zimm 1 General Comment The GAATA survey covers Part 91 and On

- Demand Part 135, while the T

- 100 data set covers Part 121 and Part 135 operators with more than $20M in annual revenue. It seems that there is both an overlap and a gap between the two data sets. Some overlap will

occ ur between GAATA and T

- 100 for large On

- Demand Part 135 operators with more than $20M in annual revenue, but neither database will cover smaller Part 135 Scheduled operators with <$20M in revenue. Some additional discussion or sensitivity analysis on the m agnitude of this missing data would be helpful for Clarifying text added

7Table 2. Comment/Response to the Draft

Commenter Page Section Text Comment Response the reader. It’s likely that the inclusion or estimation of this data will not drastically change the emissions forecast; however, scheduled Part 135 operators generally have high utilization and thus will materially contribute to total fuel burn and emissions.

Ralph Rissmiller 6 1.0 The GAATA data include all activity (by aircraft type) for general aviation and air taxis, while the T

- 100 data include commercial activity by specific aircraft make and model, which include air taxis but not non

- commercial general aviation. It appears this is intended to cover Part 91 and 135 operations, and T

100 data covers air carriers, but not totally clear. The following phrase is what creates my
qu estion. It says that T

- 100 data includes air taxis (and air carrier), but not non

- commercial general aviation. The preceding statements imply that the FAA data report includes Part 91 as well as Part 135 commercial/air taxi operations. I’d suggest clarifyi ng if possible. From the text, I got the idea that the commercial/air taxi small aircraft were included in both reference reports. Clarifying text added

8Table 2. Comment/Response to the Draft

Commenter Page Section Text Comment Response

Ralph Rissmiller 6 1.0 For turboprops, the aircraft make and model from the T

- 100 data above and below the MTOM threshold were determined using the MTOM listed for each aircraft make and model in the FAA’s Emissions Dispersion Modeling System (EDMS) system tables (FAA, 2013). There are very few non

commercial operators in the US of turboprop aircraft with a MTOM >8618 kg, with most <5700 kg. The vast majority of the general aviation turboprop aircraft are <8618 kg. Good validation of our results. No change required

Ralph Rissmiller 7 2.0 For base year 2012, the T

- 100 activity data include air time and LTO by specific aircraft make and model (FAA, 2012). It may be worth considering that aircraft utilization decreased during 2012

- 2014 due to economic concerns. SOW is limited to data availability at the time that the project was being implemented ; there w as only partial data for 2014. Variance in activity from year to year is noted in the projections section. No change required.

Ralph Rissmiller 8 2.0

- Table

2.1 Table 2

- 1. The Number of Aircraft in each Aircraft Category I believe this table needs some work. The weight numbers don’t match up with the EDMS category statements. I also question the number of models indicated for the very heavy large and medium business jets (around 400

- 7 50K lbs). There are only a very few business jets in this weight class. The vast majority are <70K lbs, with only the very largest approaching 120K lbs. It appears that a number of the large Boeing and Airbus aircraft are being included. The table categori es were check and
t he MTOM values listed in the tables represent the smallest aircraft size reported in EDMS for the aircraft categories used in EDMS. Though they do not exactly match the values used in the CO 2 standard threshold

, they are relatively close and useful for our purposes.

No change required to weight class numbers, but text was added that individual aircraft MTOM may be different than the average values presented in Table 2

- 1.

9Table 2. Comment/Response to the Draft

Commenter Page Section Text Comment Response Note Table 2

- 1 includes all aircraft, including those above and b elow the CO 2 standard thresholds. F or our purposes only helicopter and small turboprop data are needed in this analysis, the others presented in Table 2

- 1 are above the ICAO GHG threshold and included as they needed to be evaluated to determine if they a re considered air taxi’s and should be removed from the GAATA estimates

. The larger aircraft categories were removed from the table as they are unlikely to include air taxis.

Glenn McDonald/ Peter Zimm General Comment In Section 1.0, the methodology for reconciling the turboprop fleet and utilization between T

- 100 and GAATA is a bit unclear. By definition of the weight class rules, it’s likely that all (or at least the vast majority) of turboprop operations with GAATA a re from aircraft less than the MTOM threshold, while some of the T

100 turboprop data is likely from aircraft greater than the MTOM threshold. The subtraction of the T

- 100 data from the GAATA data is therefore a bit unclear. Thus, it Clarifying text added

10Table 2. Comment/Response to the Draft

Commenter Page Section Text Comment Response seems likely that the small turboprop operations were underestimated by removing more GAATA data than appropriate.

Ralph Rissmiller 8 2.0

- Table

2.1 Propeller Not clear to me what this category actually includes. Is it the reciprocating engine fleet? Added general aviation category text

Ralph Rissmiller 8 2.0

- Table

2.1 Large Business Jet

– Average MTOM This may be biased slightly towards the large business jet class (i.e., Bombardier Challenger and Global Express, Gulfstream, and larger Dassault aircraft ) . A substantial number of the smaller jet aircraft are <25,000 lbs MTOM. Noted in the text that individual aircraft MTOM may be different than the average values presented in Table 2

- 1

Ralph Rissmiller 8 2.0

- Table

2.1 Large Business Jet

– Number of Aircraft Models The only general aviation “large jet” aircraft are Boeing and Airbus aircraft, and there are only a handful in service as general aviation aircraft. These are commercial air taxis that are considered large jets in EDMS

. No change required.

Ralph Rissmiller 8 2.0

- Table

2.1 Medium Jet

– Number of Aircraft Models I am not aware of a population of 16 aircraft models that would be categorized as medium jet business aircraft in this weight class. This is approaching the weight of an A330 or B767. In general, the largest general aviation business aircraft are in the 10 0,000

- 130,000 lbs MTOM range, while the majority of the Removed from table as they are unlikely to include air taxis

.

11Table 2. Comment/Response to the Draft

Commenter Page Section Text Comment Response larger long range business jets are in the 70,000

- 90,000 lbs MTOM class.

Ralph Rissmiller 8 2.0

- Table

2.1 Medium Jet

– Average MTOM There are only a handful of large business jets in the weight categories of 758,550 and 391,760 lbs (Boeing and Airbus business jets). The vast majority of aircraft in these weight classes are large commercial transport aircraft. Removed from table as the y are unlikely to include air taxis

.

Ralph Rissmiller 8 2.0

- Table

2.1 Small Jet

– Average MTOM The average weight of 176,724 lbs does not fit the small jet category. Is this an error in labeling? It appears that the 37,567 small jet and 176,724 lbs business jet values may be reversed. Checked data, average is correct

– no change required

Ralph Rissmiller 8 2.0 Because the Quest Kodiak 100 did not have aircraft information in EDMS, a comparable value was used from a similar aircraft in EDMS. There are only a small number of these aircraft in service at this time. This aircraft is quite similar to the Cessna 208 Caravan. Surrogate changed

Glenn McDonald/ Peter Zimm 8 General Comment Appears to be a typo on 1 ton = 0.0005 lbs Correction made

Glenn McDonald/ Peter Zimm General Comment The GAATA survey should include the vast majority of the U.S. helicopter fleet, as nearly all helicopter operations are classified as Part 91 or On

- Demand Part 135. Total utilization for this fleet is available from GAATA as well. Clarifying text added

12Table 2. Comment/Response to the Draft

Commenter Page Section Text Comment Response ICF believes that the GAA TA “total hours by aircraft type” Table 3.2 is meant to represent total aircraft flying hours, not just cruising hours. Similarly, the fuel burn gallons per hour is meant to be an average per

- hour burn rate over the entire flight, not just for cruise. Thus , it seems that the “T100 and GAATA Fuel Burn and CO 2 Emissions_12_18_2014.xlsx” model might be overestimating the rotorcraft fuel burn numbers by double counting the cruise and LTO portions of the flight. The total hours and average fuel burn per hour rat es appear to include the portion of fuel burned during LTO. ICF suggests reviewing the rotorcraft fuel burn output on the “GAATA” worksheet and setting the total rotorcraft fuel burn to 509,339 tons (the current cruise figure), instead of adding in the 459 ,346 tons from the additional LTO estimation.
Ralph Rissmiller 9 2.0

- Table

2.2 Eurocopter AS350 B2 AStar SA350 TPE331

- 3 TPE3 Weight factors based engine market share Why is this the only helicopter listed? There are a large number of Hughes, Bell, Sikorsky, etc. This is the only helicopter reported into the T

- 100 dataset. No change needed

.

13Table 2. Comment/Response to the Draft

Commenter Page Section Text Comment Response helicopters in service. Also Robinson (R66) has a turbine powered version as well. Note that T

- 100 data may not include all aircraft as they only account for commercial air carriers. The GAATA data were included as they include non

- air carrier service providers such as medivac helicopters, offshore support, pipeline surveillance and traffic monito ring helicopters

Ralph Rissmiller 9 2.0

- Table

2.2 Socata TBM850 There are additional TBM models as well. These are the only models reported into the T

- 100 dataset. C heck ed that International data were included

.

Note that T

- 100 data may not include all aircraft as they only account for commercial air carriers. The GAATA data were included as they include non

- air carrier service providers which use these smaller aircraft

Ralph Rissmiller 9 2.0

- Table

2.2 Beech C99 What about the Beech 90 series (90, A90, B90, C90), 100 series (100, A100, B100) snf 300/350 series? These are the only models reported into the T

- 100 dataset. C heck ed that International data were included.

Note that T

- 100 data may not include all aircraf t as they only account for commercial air carriers. The GAATA data were included as

14Table 2. Comment/Response to the Draft

Commenter Page Section Text Comment Response they include non

- air carrier service providers which use these smaller aircraft

Ralph Rissmiller 9 2.0

- Table

2.2 Pilatus PC

- 12 I do not see the PC9 listed. These are the only models reported into the T

- 100 dataset. C heck ed that International data were included

.

Note that T

- 100 data may not include all aircraft as they only account for commercial air carriers. The GAATA data were included as they include non

- air carrier service providers which use these smaller aircraft

Ralph Rissmiller 9 2.0

- Table

2.2 Above the MTOM Threshold I do not see the Piper Meridian (single engine turboprop) listed. These are the only models reported into the T

- 100 dataset. C heck ed that International data were included

.

Note that T

- 100 data may not include all aircraft as they only account for commercial air carriers. The GAATA data were included as they include non

- air carrier service providers which use these smaller aircraft

15Table 2. Comment/Response to the Draft

Commenter Page Section Text Comment Response

Ralph Rissmiller 12 2.0

- Table

2.4 Table 2

- 4. Aircraft and Engine Specific Emission Factors from EDMS Run The table doesn’t indicate which of the aircraft are single engine and which are multi

- engine. This may be adequately addressed in the fuel consumption rate, so just an observation. See earlier comments regarding additional aircraft models not included in the table (i.e., Beech 90/200/300 series, PC

- 9, Piper turboprop, e tc.). The number of engines was not a critical component in this study as the calculations were based on individual aircraft movements, EDMS automatically accounts for different engine configurations. No change required.

Ralph Rissmiller 12 2.0 The comparable aircraft selected was EADS Socata TBM

- 700/ PT6A

- 64. I would have suggested a comparable aircraft is the Cessna 208 Caravan. Surrogate changed.

Ralph Rissmiller 13 2.0

- Table

2.5 Normally, fuel consumption is considered in lbs/hr (not lb/min), although it is a simple conversion. To calculate emissions from cruising units need to be in terms of mass, not volume. No c hange required.

Glenn McDonald/ Peter Zimm General Comment Prior to table 2.6, it might be helpful to give some more detail on how minutes were allocated between LTO and cruise (see Helicopter Fleet/Utilization Reconciliation as well) Table added of Time in Mode assumptions used

Ralph Rissmiller 16 3.0

- Table

3.1 Turbo Jet/Turbo Prop

– Up to 5,669* Should this be the “a” footnote reference instead of “*”? Correction made

Ralph Rissmiller 17 3.0

- Table

3.3 Small Business Jet 315.1 I believe this consumption rate is more typical of the larger business jets (i.e., >30K lbs MTOW. The smaller business jets (like the Observation noted in the text

16Table 2. Comment/Response to the Draft

Commenter Page Section Text Comment Response Citation 500 series which are around 12

- 25K lbs) would more likely average around 1

, 000 lbs/hr (or approx. 125 gph).

Ralph Rissmiller 17 3.0

- Table

3.3 Helicopter 54.7 Does this include all helicopters or only the AStar listed above? Also, not clear if this column is per engine or per aircraft (i.e., single or multi engine). I realize this may not be a factor, since I believe fuel consumption is being addressed as per airplane. If so, then it is addressed that way. Clarification made to title

Ralph Rissmiller 19 3.0

- Table

3.5 Table 3

- 5: 2012 T

- 100 and GAATA Combined Fuel Use and CO2 Emission Estimates by Aircraft Type and Mode Need to update table with T

- 100 international Segment Update included

Ralph Rissmiller 19 4.0 4.0 Comparisons Under development Comparison included

Ralph Rissmiller 19 4.0 Parts of the comments below the table are not visible. Table will be replaced

Ralph Rissmiller 20 5.0 To estimate anticipated future emissions from small business jet, turboprop, and helicopter activities, projected general aviation fuel usage data were obtained from the FAA’s Aerospace Forecast, 2014

- 2034 (FAA, 2014b). I recognize this is an FAA publicat ion, however I believe the projected growth is rather large considering the current fleet growth rate. The projections in the report are consistent with the projections in the FAA Aerospace Forecast

. No change needed.

Ralph Rissmiller 20 5.0 Table 5

- 2 summarizes projected CO 2 emissions for small business jets, turboprops, and helicopters, based on the 2012 CO 2 emissions and the growth factors Should this be Table 5

- 1? Correction made

17Table 2. Comment/Response to the Draft

Commenter Page Section Text Comment Response included in Table 5

- 1.

Ralph Rissmiller 22 6.0 Assessed the fraction of CO 2 emissions associated with the aircraft relative to total jet CO 2 emissions in the EPA Emission and Sinks report and Fast Facts: National

- Level U.S. Greenhouse Gas Inventory (EPA, 2014b). Pending inclusion of international segment of T

- 100 data. Com pleted

18