INTERGOVERNMENTAL PANEL ON CLIMATE CHANGE

IPCC/OECD/IEA PROGRAMME ON NATIONAL
GREENHOUSE GAS INVENTORIESManaging Uncertainty
in National Greenhouse Gas Inventories

Meeting Report

Paris, France
13-15 October 1998

Supporting material prepared within the IPCC/OECD/IEA National Greenhouse Gas Inventories Programme for consideration by
the Intergovernmental Panel on Climate Change. This supporting material has not been subjected to formal IPCC review
processes.

Acknowledgements
This report was prepared by Jim Penman, Dina Kruger, and Ian Galbally with assistance from Pierre
Boileau and Thomas Martinsen. Additional contributions were received from Roberto Acosta, Simon
Eggleston, Jos Olivier, Carmen Benkovitz, and Jean-Pierre Fontelle. We are grateful to the sixty two
inventory scientists and inventory experts (including eleven from developing countries and twelve
from international organisations) who contributed to this report by attending the workshop, and to
Lee Beck, Pavel Fott, Katarina Mareckova, Jeoren Meijer, Newton Paciornik, Tinus Pulles, Kristin
Rypdal, and Amy Emmert who commented on the draft. We also thank members of the secretariat,
including Karen Treanton, Yamil Bonduki and Béatrice LeMen. .

Table of Contents
1. EXECUTIVE SUMMARY ...............................................................................................................................4
2. INTRODUCTION .............................................................................................................................................5
The Policy Context..............................................................................................................................................5
Science and Uncertainty .....................................................................................................................................6
3. THE CURRENT STATE OF UNCERTAINTIES AND GOOD PRACTICES IN NATIONALINVENTORIES .....................................................................................................................................................7
3.1 METHODOLOGICAL ISSUES IDENTIFIED BY THE UNFCCC SECRETARIAT ............................................................7
3.2 NATIONAL CASE STUDIES...................................................................................................................................7
Built-in transparency (Australia)........................................................................................................................7
Changing emission factors (United States).........................................................................................................8
Challenges for developing countries (China) .....................................................................................................8
Uncertainty and the Emissions Trend (United Kingdom) ...................................................................................9
Effect of inventory uncertainty on trend (Norway) ...........................................................................................11
4. TOOLS AVAILABLE TO MANAGE AND REDUCE UNCERTAINTY .................................................13
4.1 APPROACHES TO IMPROVE INPUT DATA ............................................................................................................13
Emission factors and emission calculation methods.........................................................................................13
Activity statistics ...............................................................................................................................................13
4.2 QUALITY ASSURANCE/QUALITY CONTROL (QA/QC) AND AUDITING (VALIDATION) .......................................14
4.3 INVENTORY REVIEW ........................................................................................................................................14
Expert (Peer) Review ........................................................................................................................................15
Stakeholder Review...........................................................................................................................................15
Public Review ...................................................................................................................................................15
Central Review..................................................................................................................................................15
4.4 CHECKING THE INVENTORY (VERIFICATION)....................................................................................................16
Inventory Comparisons.....................................................................................................................................16
Atmospheric Verification ..................................................................................................................................16
5. HOW CAN UNCERTAINTIES BE EVALUATED AND COMMUNICATED?......................................17
6. HOW CAN WE MINIMISE BIAS? ..............................................................................................................19
Completeness ....................................................................................................................................................19
Use of Appropriate Methodological Approaches .............................................................................................19
Inventory Quality Assessment ...........................................................................................................................19
Transparent Reporting......................................................................................................................................19
Identification of Priority Areas .........................................................................................................................20
7. WHAT FURTHER WORK DOES IPCC NEED TO ASSESS AND CO-ORDINATE? ..........................20
ANNEXES............................................................................................................................................................21
ANNEX 1: GLOSSARY OF TERMS ...........................................................................................................................22
ANNEX 2: GOOD PRACTICE IN INVENTORY PREPARATION ....................................................................................28
ANNEX 3: ISSUES FOR SECTORAL WORKSHOPS TO CONSIDER...............................................................................32
ANNEX 4: DESCRIPTION OF THE UNCERTAINTY FRAMEWORK ...............................................................................36
ANNEX 5: LIST OF PAPERS PRESENTED AT THE MEETING......................................................................................43
ANNEX 6: LIST OF PARTICIPANTS ..........................................................................................................................45

1. Executive Summary
Although the IPCC 1996 Revised Guidelines do provide some advice on managing uncertainties ingreenhouse gas emissions inventories, the size of the uncertainty ranges quoted by the IPCC, as wellas the prospect of legally binding targets under the Kyoto Protocol, suggest the need for further
progress on this issue.
At its eighth session, the Subsidiary Body on Scientific and Technological Advice (SBSTA) asked
IPCC to “...give high priority to completing its work on uncertainty, as well as to prepare a report on
good practices in inventory management and to submit a report on these issues for consideration by
the SBSTA, if possible by COP 5.”
This request formed the basis for the Paris meeting, at which national inventory experts and the
UNFCCC Secretariat gave presentations on experiences with the quantification of uncertainties ininventories and on options for managing uncertainties in the future.
Present evidence suggests that the uncertainty in GWP weighted emissions in any year could be
20%1 or more. However, the difference in emissions between years is likely to be known moreaccurately than this because the errors in base year and in subsequent years are likely to becorrelated. Nevertheless the uncertainty in the difference between the base year and the firstcommitment period may be comparable to the typical difference between base year emissions and theassigned amount for Annex I Parties.
Despite better scientific knowledge uncertainties of this magnitude in inventory estimates and trends
are likely to remain between now and the first commitment period.
If uncertainties cannot be eliminated, they can be managed, using good practice procedures in theselection of emission factors, methodologies, and activity data. These procedures are designed toreduce uncertainties and minimise any bias. In this case, the values recorded (although uncertain)
would be the best available estimates of actual emissions, and therefore presumably the best available
basis for assessing compliance with commitments under the Convention and the Protocol. The ideaof good practice also extends to ensuring transparency for review purposes, to evaluating
uncertainties, and to cross checking national estimates with independent calculations and empirical
data. Good practice recommendations from IPCC could be considered by the Parties forincorporation into the guidelines for national inventory systems to be decided under the provisions ofArticle 5 of the Protocol.
The use of existing IPCC 96 Guidelines is required under the Kyoto Protocol. However, theGuidelines give great flexibility to Parties in choosing inventory methodology and therefore their use
is not itself sufficient to guarantee that good practice has been used.
In order to define good practice within the context of the existing IPCC 96 Guidelines, the Paris
meeting proposed that IPCC should organise four sectoral workshops dealing with good practice ininventory estimates, one workshop dealing with cross cutting issues on the quantification andmanagement of uncertainties, and a concluding workshop to ensure consistency in the definition ofgood practice as a whole. These workshops would be held in 1999 and 2000 and be scheduled inorder to make a substantive report on progress available as soon as possible.
1 Estimated as two standard deviations divided by the mean.

42. Introduction
The Expert Group meeting on Managing Uncertainty in National Greenhouse Gas Inventories (Paris
13-15 October, 1998) was held as a continuation of the programme of work defined at IPCC XIII
(Maldives, 19-25 Sept. 1997) and following the request from the 8th meeting of SBSTA in Bonn inJune 1998. In its request SBSTA noted that the Kyoto Protocol includes provisions related togreenhouse gas inventory methods, and encouraged the IPCC/OECD/IEA Inventories Programme to:“...give high priority to completing its work on uncertainty, as well as to prepare a reporton good practice in inventory management and to submit a report on these issues forconsideration by the SBSTA, if possible by COP5.”

The Policy Context
COP5 will be held in October/November 1999, and will be used for reporting progress on the IPCC’s
programme of work discussed during the Paris meeting. Beyond COP5, Article 7 of the Kyoto
Protocol requires that decisions on use of inventories to demonstrate compliance with commitments
under the Protocol be made by the Conference of Parties serving as the Meeting of the Parties to the
Kyoto Protocol at its first session (COP/MOP1). Article 5 of the Protocol also requires the
COP/MOP1 to make decisions on national systems for inventories. These systems may well coverissues relevant to managing uncertainties, such as arrangements for transparency and quality control.
The work programme agreed in Buenos Aires means that preparatory work for the COP/MOP1
decisions will be completed by COP6, planned for October 2000. In order to input effectively to
COP6, IPCC ought to have completed its work early in 2000 to allow for the necessary review
process prior to the meeting of SBSTA in June 2000.
The Kyoto Protocol goes beyond the Framework Convention on Climate Change in introducing
legally binding commitments on emission levels. This legally binding aspect explains SBSTA’s
increased interest in inventory management and uncertainties. Also, the prospect of using flexible
mechanisms, including emissions trading, means that Parties will have even greater interest in thereliability of other national inventories.
In Kyoto, the Parties recognised that greenhouse gas inventories are uncertain, and that unlessuncertainties are reduced and managed, there is a risk that Parties could adjust their emissions
estimates within the band of uncertainty to help them “meet” their commitments, introducing bias
into the emission estimates.
Article 5 of the Protocol, taken together with paragraph 1 of Decision 2/CP.3, says that the IPCC
1996 Revised Guidelines shall be used, and that if this is not done, appropriate adjustments agreed at
COP/MOP1 shall be applied. The mandatory nature of this requirement underlines the importance of
reducing uncertainty.
The IPCC 1996 Revised Guidelines (henceforth, “Guidelines”) contain a detailed list of source
categories, but they are less prescriptive about calculation methods. In fact the Guidelines encourage
Parties to use their national methods and assumptions where these are more accurate than themethods described in the Guidelines. This has led to the idea of formulating good practice guidance
to supplement the procedures currently outlined in the IPCC Guidelines for the purposes ofcalculating emissions. Good practice can refer not only to estimation methods, but also to the way inwhich inventories are managed, including the management of uncertainty.
IPCC’s role is to provide advice to governments that is policy relevant, but not policy prescriptive.
With this in mind, IPCC can help the Parties make the decisions at COP/MOP1 called for under
Articles 5 and 7 of the Protocol by:5a) providing quantitative assessments of the implications of uncertainty in greenhouse gas
inventoriesb) defining how good practice guidance could be used to reduce uncertainties and minimise bias
in emissions estimates.
On a) uncertainty in emission factors, model parameters, and activity data mean that Parties would
have (potentially at least) the possibility of choosing values within the uncertainty ranges to help
them appear to meet their commitments. The good practice guidance under b) would aim to remove
this possibility by making the inventory estimates as reliable as possible.
How prescriptive the good practice guidance should be will of course be for the Parties to decide, but
the IPCC can assist in this process by providing the necessary technical information on what is
feasible. This could include selection and application of estimation methods and management of
inventories in terms of quality control and review procedures to ensure unbiased estimates. The IPCC
could also help prioritise efforts to reduce uncertainties.
The Paris meeting heard expert presentations relevant to both a) and b). The report which follows
summarises the existing state of knowledge based on these presentations, and sets out plans andagendas for a work programme of six sectoral and methodological workshops to be held over the
next eighteen months to improve the quantification of uncertainties and define good practice. This
work is intended to provide substantive reports to COP5, and any subsequent COPs up to
COP/MOP1, in accordance with the request from SBSTA8, the Buenos Aires workplan and therequirements of the Protocol.

Science and Uncertainty
Uncertainty in greenhouse gas inventories stems from limitations in our knowledge about theprocesses which generate emissions and uptakes. Limitations relate to:•the state of knowledge about the variety of processes which generate emissions;•the number of measurements available to develop models or algorithms for the
calculation of emissions and uptakes; and•the resources available for collecting the statistical data necessary for the preparation of
inventories; and•the intrinsic statistical variability of the processes leading to emissions.

For the scientific community, estimation of uncertainty in national inventories is important forassessing how anthropogenic emissions are contributing to changes in atmospheric concentrations ofgreenhouse gases. Appropriate quantification of uncertainty can provide the basis for prioritisingscientific research needed to provide more accurate emission estimates. Furthermore, improved
emission inventories and measurements of atmospheric concentrations are essential for assessing theimpacts of anthropogenic greenhouse gas emissions on the climate system.

6

3. The Current State of Uncertainties and Good Practices in National
Inventories
Assessing existing national approaches to managing uncertainty and inventory quality is importantfor understanding the current state of national inventories and defining where additional guidance isneeded. This section covers issues identified by the UNFCCC Secretariat in analysing inventory datafrom national communications, and presentations made by national experts covering (1) institutionalframeworks for producing greenhouse gas inventories, (2) ways of revising methods and data, (3)efforts to produce greenhouse gas inventories in a non-Annex I Party, and (4) analyses of
uncertainties in a particular year and the uncertainty in the emissions trend.

3.1 Methodological issues identified by the UNFCCC Secretariat
In the compilation of inventory data submitted by Annex-I Parties (FCCC/SBSTA/1998/7 and
FCCC/SBSTA/1998/8), the UNFCCC Secretariat identified several issues that are relevant to goodpractices and uncertainties.
All Parties except two reported significant changes in their estimates by changing their methods,
emission factors or activity data between the first and second national communications. Changes
ranged from 0.2 to 38 percent for 1990 estimates. Variations in methods and data used can limit
comparability over time and, perhaps, introduce an additional source of uncertainty.
Parties, however, did not report uncertainties in a consistent or comparable way, nor did they discuss
the changes in uncertainty over time, perhaps due to difficulties in estimating uncertainty and/or
incomplete guidance on these matters in the IPCC and UNFCCC Guidelines (FCCC/CP/1996?15
Add. 1 decision 9/CP2).
There were only small changes in the mix of gases for most Parties over the period 1991-95 using
1990 as a base year, with CO2 changing by an average of 2%. The change in weighted-average
uncertainty due to the change in the mix of gases is about 3.2% on average for the 30 Parties
considered. This is less than the absolute uncertainty in any given year. However, this contribution to
uncertainty in Parties’ inventories is not likely to be cancelled out by correlation of uncertainties
between the base year and the commitment period. Furthermore, change in the trend which can be
introduced due to the changes in methods were larger than 5% in 16 of 34 Parties. This was based on
a comparison of the percent change over the period using second national communication methods
with the percent change using first national communication methods.
Completeness of reporting by Parties was difficult to determine because the notation key (NA, NO,
IE, NE, 0) provided for in the IPCC methodology terminology are not being used consistently.
Clearer definitions of such terms may be needed to avoid different interpretations, and to ensure that
Parties report their inventories in a transparent, complete and consistent way.

3.2 National Case Studies
Built-in transparency (Australia)
The process of greenhouse gas inventory development in Australia began in 1993. Several Inventory
Methodology Working Groups were established, consisting of representatives from government,
industry, research, academia and NGOs. These groups began developing methodologies, sector by
sector within the context of the IPCC Revised Guidelines, by obtaining opinions from their respective
constituent organisations. A broad spectrum of views was gathered and a consensus reached on the7methodology to be used. Each methodology was then subjected to public review and required theendorsement of the National Greenhouse Gas Inventory Committee (NGGIC).
The procedure for making changes to the inventory methodologies requires publication of working
papers by researchers, documentation of upgrades and a round of peer review. Following this, themethodology must be submitted for endorsement by the NGGIC.
The inventory then undergoes wide distribution before publication, to obtain input from the user
community. The requirement that the inventory be communicated in this way helps ensure thetransparency of the inventory. Along with the main inventory, there are Workbooks published foreach of the methodologies. These include the rationale, assumptions, methodologies, emission
factors and worked examples used to estimate the emissions. These procedures are used to minimise
the uncertainty in the inventory and maximise its accuracy for its intended purpose. This results in an
inventory which is considered credible and fulfils the needs of its users.
Even with this framework in place, there are still challenges in preparing the inventory. Areas ofsignificant uncertainty still exist, and some basic activity data for calculating the emissions estimates
may be collected infrequently. However, with rigorous documentation and continuing inventory
development, these changes can be made transparent to the Convention and other Parties.

Changing emission factors (United States)
In 1998, during the public review period for the Draft Inventory of US Greenhouse Gas Emissions
and Sinks, the Office of Mobile Sources (OMS) of the US Environmental Protection Agency
conducted an evaluation of whether the emission factors for mobile sources used in the inventory
were appropriate. The emission factors in question were taken from the Guidelines. Their evaluation
identified that these emission factors were obtained from very little test data (5 vehicles on European
testing cycles), and in one case the emission factor was based on an assumption rather any actual test
data. Following this evaluation, it was decided that a measurement programme should be undertaken
to develop more appropriate emission factors for use in the inventory.
The programme undertaken by USEPA focused on determining the emissions profile of arepresentative fleet of US vehicles. Several interesting conclusions resulted, including:•emissions of N2O increased with fuels of higher sulphur content;•

N2O emissions were higher with air conditioning on;•

emissions of N2O did not vary significantly with the mileage of the vehicle; and•light-duty trucks had significantly higher N2O emissions than passenger vehicles.

This was a limited testing programme and therefore there were several recommendations for further
work. These included increasing the testing of heavier gasoline vehicles, studying the effect ofsulphur content on N2O emissions and studying the reasons for the large variability of N2O emissions
in routine testing.
In this case, the process of public review allowed a weakness in the inventory to be identified andaddressed. Since the IPCC emission factors for N2O were determined to be inappropriate for the US
inventory, a targeted measurement programme was used to obtain new emission factors and improve
the accuracy of the inventory.

Challenges for developing countries (China)
Three national inventories of CO2 emissions have been produced in China in recent years. Resultsfrom these show a variation of almost 25% in the results. Apparently, much of this variation is a
result of changing methodologies and the choices made for the activity data and emission factors.

8This variation could be an indication of the current uncertainty of the inventory for China. The
inventory practitioners in China have identified areas where particular problems exist:•

the estimate of carbon stored in products varied by 10% among the three inventories;•

measurements of the oxidation rates of fuels in China showed variations of 8-11% fromthe IPCC default factors;•

the emission factor for CO2 from cement production varied by more than 25%;•

expert judgement was used to estimate emission factors for fugitive emissions.

These difficulties are a result of the lack of a unified, systematic and comparable methodology inChina. Moreover, the large geographic areas concerned and the variability of practices in various
regions can make default emission factors inappropriate. Future efforts will try to use a more
systematic and comparable approach, including reconciling Chinese energy statistics with those ofinternational data sources so they are more compatible with IPCC methodologies. Combustion
technologies in China are expected to change rapidly and therefore ongoing efforts to provide basic
data will be required to keep the inventory up-to-date. These are particular challenges to producing
accurate inventories in rapidly changing economies.

Uncertainty and the Emissions Trend (United Kingdom)
The trend in emissions between a base year (e.g. 1990) and a target year (e.g. 2010) is critical forassessing how countries are meeting targets, both under the Convention and under the Protocol. So
we need to analyse not only the uncertainties in a particular year, but also how uncertainties in theemissions of different gases and sectors combine over time. To do this we need to combine
uncertainties which can vary widely by gas and by sector.
It is difficult to do this by classical statistical methods, so the UK has used so-called Monte-Carloanalysis (see box). Monte Carlo analysis is equivalent to classical methods but is better able to handle
uncertainties which are large compared to the mean. In addition, this type of analysis can combine
empirical quantitative data on uncertainties with expert judgement where empirical data are sparse,and can be set up to handle correlation of
uncertainties over time.
Monte Carlo Analysis (MCA) is a numericalmethod to estimate overall uncertainty for
For this study each emission source was processes having several stages, each of which areinvestigated and the uncertainty in the uncertain. Each uncertain quantity is specified by aemission factors and activity rates was probability distribution function (PDF) which,estimated. This information was derived from ideally, is derived from empirical data, but can bea number of sources, for example:
specified by expert judgement if insufficient
empirical data are available.
• where a large number of measurements
were available, the probability distributionof emission factors were derived directly;•

statistical differences in energy statistics
gave an estimate for the uncertainties for
activity data on fuel combustion; and•variations in calorific value of fuels gave
an indication of the variability in emission
factors.

The results are summarised in Table 1. In thisTable uncertainties in 1990 and 2010 are

The overall uncertainty in the process is determinedby running the Monte Carlo simulation many times,
allowing uncertain quantities to take random values
within their PDFs. The result is a PDF for the wholeprocess which is consistent with the individualcomponent PDFs.
MCA gives the same results as classical statistics
where the PDFs can be combined analytically, andis better able to deal with PDFs that are broad incomparison with the mean. Nevertheless, whereexpert judgement is involved, the results will only
be reliable if the judgement is sound.
9

shown as two standard deviations divided by the mean, expressed as percentages. This shows
approximately 95% confidence limits provided the uncertainty distribution is normal. The actual
2.5th and 97.5th percentiles are shown for the change in emissions between 1990 and 2010. This
range shows the 95% confidence limits even for asymmetrical distributions. Since the UK study was
about uncertainties related to inventory methods, no additional allowances for uncertainties arising
purely from projection methodologies were made. The projected activity data were assumed to be
subject to the same statistical uncertainty as the historical activity data.

Table 1: Estimates of uncertainties in the UK greenhouse gas inventory% uncertainty 1 inabsolute value of emission
estimate
1990

Change between 1990 and 2010
in absolute value of emission
estimate

2010

Gas
th

2.5 percentile
CO2 from fuel use only
CO2
CH4
N2 O
HFC
PFC
SF6
GWP weighted total2
Sinks

2%
4%
17%
234%
25%
20%
13%
19%
38%

1%
4%
17%
301%
9%
13%
13%
19%
42%

1%
-4%
-50%
-63%
145%
-72%
50%
-11%
-18%

97.5th percentile
4%
6%
-19%
-14%
311%
-56%
113%
-2%
166%

1

Note: Uncertainty is defined in cols 2 and 3 as twice the standard deviation divided by the mean. The larger
uncertainties reported in this table come from non-Gaussian (asymmetric) distributions. In these cases the uncertainties
defined in this way are indicative, and do not have the usual interpretation in terms of confidence limits, and the probabilityof actual emissions having a particular value would need to be obtained by consideration of the full probability distributions.
These can be found in the research report from the UK .
2 Using IPCC 1995 GWP values,.and excluding sinks.

Some of the emission estimates had a very large uncertainty and the combined uncertainty on total
Table 1).
GWP weighted emissions in 1990 or 2010 is just under 20% (columns 2 and 3 ofHowever, the emission estimates in 1990 and 2010 are correlated to the extent that the same
methodologies and factors are used for both years. This reduces the uncertainty in the trend relative
to the uncertainty on total emissions in any year. In the UK’s case the last two columns in Table 1
show that the change between 1990 and 2010 in GWP weighted emissions is projected to lie
between -2% and -11% with 95% confidence (i.e. a reduction in emissions of 6 ± 5%). Therefore theuncertainty in the trend (± 5%) is less than the uncertainty in the absolute level of emissions in any
year (20%2). The trend uncertainties will tend to be reduced in this way, so long as methodologies are
consistently applied over time and that the emissions mix and underlying technologies do not change
radically. Where these conditions do not hold the trend uncertainties may be greater than theuncertainties in a single year.
While this example is for the UK alone, the lessons could be more widely applicable.
•

The uncertainties in some of the gases can be very large (greater than 100%, expressed as
two standard deviations divided by the mean) but the overall uncertainty in total

2 This value is expressed as 20% rather than ± 20% because it is found to be asymmetrical.

10emissions, combined using weighing of the Global Warming Potential (GWP) for the six
gases, is much smaller (20% for the UK).
•

While more studies of this kind are needed, it is possible that, for the majority of parties,
the trend in GWP weighted emissions will be better known than the absolute value (i.e.
will have lower uncertainty, estimated to be ± 5% for the UK).

•

The uncertainty in the change in GWP weighted emissions, (± 5% for the UK), isnevertheless significant compared to commitments for emission reductions.

These results imply that methodological choice in inventory calculations within the framework of the
IPCC Guidelines would give Parties, in principle, the ability to adjust the trend in emissions between
1990 and the first commitment period under the Protocol by several percent.
This type of uncertainty analysis could be performed by more Parties, which could lead to better
understanding of the relationship between uncertainties and compliance assessments. It would also
help prioritise efforts to reduce uncertainties. However, despite increasing scientific effort, it isunlikely that this inventory uncertainty will be substantially reduced in the near future. Therefore
other measures to improve and monitor inventory quality are essential to ensure that all Parties canhave confidence in the results produced by the emission inventories.

Effect of inventory uncertainty on trend (Norway)
Uncertainty in the emission estimate in the Norwegian greenhouse gas inventory was assessed bythorough expert judgement and allocated to the IPCC recommended quality categories (high, medium
or low uncertainty). As a first step to refine the estimate of uncertainty, expert judgement and the
IPCC classification were applied to the activity data, the emission factor or direct emission
measurement in each of the IPCC source categories. This resulted in an estimate of uncertainty intotal (GWP weighted) greenhouse gas emissions at about + 10-15 %, and also identified some of themain weaknesses in the inventory. This assessment of uncertainty in emissions level however, did notgive any direct information on the uncertainty in the trend.
Uncertainty in the emissions trend was assessed by evaluating the variation required in each gas tochange the trend by a pre-determined percentage. Norway chose two cases evaluating the sensitivityof 1 % and 1/2 % respectively. The results confirm the weaknesses found by estimating theuncertainty in the individual sources, and they further show that if the trend of a single gas is very
different from the trend of the major gases it may have a large influence on the uncertainty in thetrend. This is even true for gases that constitute a small part of the total greenhouse gas emissions in
Norway. For the Norwegian inventory, the effect of an error of 100 % in the level of SF6 introduces achange of one percent in the total greenhouse gas emission in 1996, while only +/- 24 % error isneeded to give a 1 % change in the trend from 1990 to 2010.
Sensitivity analysis is a simple approach to identify the areas of the inventory which significantly
influence the trend. The method is objective, requires only a small amount of resources, and may be
applied to historical data as well as to projected emission values.
The main variables to perform a sensitivity analysis of errors in the trend can be combined using thefollowing formula, which assumes the same proportional error in the base year as in future years:∆t = xs1µs (as − t ) / ( x1 + xs1µs ) ,11where the quantities used in the equation have the following meaning:
x1
t
xs1
as

µs

original total greenhouse gas emissions estimate in year 1 (the base year)
trend of the total estimate, ie the difference between emissions in a future year and year 1,
divided by emissions in year 1
original emissions sub-estimate of source or pollutant s in year 1
trend of original sub-estimate of source or pollutant s, where the trend is defined as for terror factor of the sub-estimate of source or pollutant s, assumed to be the same in the baseand future years, defined such that if (for example) the sub-estimate changes by say +10%,
then µs would be +0.1.

The error on the trend will of course be affected if the proportional error of individual gases changeover time and formulae can also be developed to cover these cases also.

12

4. Tools available to manage and reduce uncertainty
The national case studies presented above and the assessments by the UNFCCC Secretariat suggestthat the absolute uncertainty in national inventories in a given year can be 20% or more at about the
95% confidence limit. They also suggest that the uncertainty in the trend, though less than this, isstill likely to be significant compared to the level of commitments. This indicates that, in the absenceof further action, Parties would (potentially at least) have the ability to use methodologicaladjustments, within the range of the existing IPCC Guidelines, to help them appear to meet theircommitments. But to adjust an inventory in this way would be to bias it for a political end, and thiswould not be scientifically acceptable.
It is unlikely that scientific advances will greatly reduce these uncertainties in the near future.
Therefore Parties may wish to consider the requirements for additional advice (within the context of
the IPCC Guidelines) to help ensure lack of bias in national inventories. This advice could bepresented in the form of good practice guidance.
This section describes some elements of good practice that are already in place in some countries.

4.1 Approaches to improve input data
Emission factors and emission calculation methods
Targeted measurement programmes can produce more representative emission factors for a Party,
and these values may be more appropriate for a particular Party than the IPCC defaults; but thischoice should be documented, explained and reviewed. Currently, several Parties base their emission
estimates on empirical data from measurement programmes undertaken in their own countries orregions. The emission factors that result are used in preference to the default factors in the IPCC
Guidelines. This is likely to increase the accuracy in the emission estimates but could lead toproblems of comparability among Parties unless there is an open and transparent way of
independently checking and validating the factors.
In some inventory sectors or sub-sectors, such as emissions of CO2 from fossil fuels, emissions areclosely related to the carbon content of the fuels and so measurement of carbon content may be more
appropriate than direct emission measurement. Since carbon content is correlated with heat content,
this property of fossil fuels could be used to check the reasonableness of the emission factors.
A central register of emission factors held by the UNFCCC (or another appropriate body) would
enable consistency and comparability checks of emission factors to be made. Countries that use
different factors would need to explain its source and why they have chosen it over the default value.
This would also be a valuable resource for other inventory compilers.
Sometimes alternative methodologies for estimating emissions are developed as a result of
measurement programmes. This may be appropriate where it leads to a clear improvement in theestimates. National methods are being developed to make the best use of available national statistics.
For example, in some countries there are detailed models for estimating emissions from roadtransport. These were developed based on national vehicle measurement programmes and areadapted to the national transport data. The different metnods take into account the differing
regulations among countries. In some cases the national and IPCC methods may be equivalentmathematically, and in this case the difference between them would be apparent rather than real. In
all cases national methods should be transparent and preferably peer reviewed.

Activity statistics
The accuracy and reliability of activity statistics varies widely. Fuel consumption data is usuallyregarded as reliable in most Annex-I countries, since fuels are an economic commodity. But even13

here problems can arise; for example, the division between domestic aviation and international
aviation bunker fuels is usually not well defined.3 For some other sectors the activity statistics may
be of much poorer quality, for instance waste statistics may not be collected on an annual basis or
may be incomplete, land use change statistics are often only collected every few years andcorrelations must be made to estimate yearly data, and statistics on some agricultural practices also
may not be collected regularly. These types of statistics are often obtained from research studies
which appear only once in the literature. There is need for development and dissemination ofknowledge about the procedures used to determine uncertainty in ‘single collection’ statistics activity
where repeated measurements are impractical. The common methods of determining uncertainty
used in physical sciences (determining precision by repeated measurement) are not applicable inthese cases. Depending on the importance of these sources, specific programmes could be developed
to improve the activity statistics. In some cases, however, it is possible to cross-check activity data,
for example, between satellite and ground based data on land-use change or between overall energybalances where international data collection occurs.

4.2 Quality Assurance/Quality Control (QA/QC) and Auditing
QA/QC procedures, such as those embodied in the ISO9000 series or described in guidance
documents for other types of emission inventories, provide standards for documentation and external
audit so that calculations can be checked. This ensures that each number used in the inventory is
traceable to its source. This is a valuable process, although, of course, it does not ensure that the best
numbers and factors are being chosen. Some country inventory processes follow international
standards such as ISO9000 (e.g. the United Kingdom and Netherlands); others are working to achievethis. In principle, an inventory could be accurate without QA/QC, however it would be impossible todemonstrate this accuracy, given the amount of data needed, the range of data sources, the number ofcalculations and the revisions to the base data that occur. One part of these standards is auditing byaccredited organisations. However, the auditing required for inventories might be more extensive
than that required by typical QA/QC standards and could include checking of methods by experts
which is not provided for by ISO9000.
An extensive inventory QA/QC procedure might have two phases. Firstly, QC activities would beundertaken during compilation of the inventory in order to:•provide routine and consistency checks and documentation points;•

identify and reduce errors and omissions;•maximise internal consistency within the inventory preparations and documentation
process; and•facilitate internal and external inventory review processes.

Secondly, QA activities such as independent review and audits, would be performed after
compilation of the inventory in order to check the internal QC objectives. If the inventory has an
acceptable level of QA/QC, it could then be certified as such. ISO9000 certification is available forthose institutes that comply with the it, but there is no standard specific to emissions inventories.

4.3 Inventory Review
Review goes beyond the QA/QC procedures needed for transparency because it also checks issues
such as choice of methodology, data and time series consistency. Four levels of review could be used
for greenhouse gas inventories. These are outlined below.

3 Only emissions from domestic aviation are included in national totals.

14

Expert (Peer) Review
Expert review of the procedures for estimating emissions is already used by some Parties to help
ensure that the methods and factors used accurately represent the particular national conditions, thatthe methods are as rigorous as possible, and that the data and assumptions used reflect the best
available information. They are usually undertaken for individual sectors and sub-sectors, specificsource categories or gases and may be very detailed. Expert review also looks beyond calculational
aspects of emissions estimation and considers the scientific and logical applicability of emission
factors and activity data. Often a detailed peer review may be undertaken at intervals (e.g., when amethod is first adopted or revised) and subsequent inventories using the same methodology or data
sources may be subject to less rigorous reviews. Some expert review is conducted when emissions
information is published in peer reviewed literature or at international conferences, but this tends to
be informal and does not cover entire inventories.

Stakeholder Review
Some countries involve stakeholders in the review of the inventories. In the Netherlands, extensive
consultations are undertaken with industrial groups to arrive at an acceptable estimate. In Australia,
there is an established process of consultation as described in Section 3.2. In the United Kingdom,
contacts with industrial organisations and large companies are used to collect data and improvemethodologies. This process will be put on a more formal footing in the United Kingdom with theestablishment of an Industrial Inventory Improvement Group specifically to collect information frominterested parties and to provide a forum for review of the methods used. A fourth example is theUnited States, where extensive contacts with industry, industrial organisations and air pollution
consultants has been developed through the development of their Compilation of Emission factors(AP42), the Emission Inventory Improvement Programme (EIIP) and the voluntary climate
programmes. Proposed factors, methods and background documents are made available to the public
for comment and review. As with expert review, this process frequently focuses on specific source
categories or gases and may not be required annually. It can overlap with peer review, as discussed
above.

Public Review
Some countries make their entire inventories available for public review and comment. In suchcases,, the draft inventory is published in a public register and the general public is given a certain
amount of time to comment on any aspect of the inventory. All comments are maintained in a publicrecord, along with the responses made to them. In some aspects, this type of public review canoverlap with expert or stakeholder review. However, there is the potential for a broader range ofcomments and issues to be raised by organisations or industry groups outside the main inventory
process.

Central Review
Central review, such as the current in-depth reviews carried out by the UNFCCC Secretariat, isanother review method. This type of review checks the comparability and consistency of national
inventories with the IPCC or UNFCCC Guidelines. It can provide a valuable feedback to Parties’ on
their understanding and use of the Guidelines and Reporting Instructions and aid the Parties and the
UNFCCC Secretariat in understanding the limitations of the data provided by the Parties. Central
review cannot substitute for detailed domestic review processes, because the reviewers may not havetime to fully review the inventory or assess all the methodological issues.

154.4 Checking the Inventory
Inventory Comparisons
Verification refers to checking the inventory against estimates complied independently, e.g., using
different methods, or by international organisations, or against atmospheric measurements.
Inventory comparisons can help the expert review process by identifying of areas for more detailed
analysis. For example, the IPCC Guidelines specifies both a Reference Approach based on energybalances and a Detailed Approach based on detailed fuel usage and activity data for calculating CO2
emissions from fuel combustion. The Reference Approach should give a figure that is within a few
percent of the detailed calculation and the trend should agree more closely. Of course the two
estimates are not independent of each other, as they will be based on the same fuel use statistics.
However, the Reference Approach does check the probable correctness of the calculations, and isfeasible for third parties to conduct.
Inventory totals can also be compared to data sets assembled by international organisations, forexample, the Global Emission Inventory Activity (GEIA), the International Energy Agency (IEA),
the Carbon Dioxide Information Analysis Centre (CDIAC) and the World Resources Institute (WRI),
which have emission estimates for a number of key atmospheric pollutants.
It may be helpful for cross-checking if energy use were to be reported with the inventory on adisaggregated basis. In sectors where non-CO2 emission estimates are based on statistics other thanenergy, for example where road transport emissions may be estimated from vehicle kilometres, thenenergy use can be the basis of an alternative emission estimate to provide a cross-check. Anotheroption is to compare emissions per capita, per unit GDP or other suitable statistic across countries.
Differences should be explicable in terms of economic structure; e.g. countries with large energy
contributions from hydroelectricity or nuclear power will have lower emissions per capita. Countries
with large car ownership will tend to have higher emissions of traffic related pollutants. This type ofanalysis tests whether the inventories are broadly consistent with the relative socio-economic
conditions of the Parties, but it does not test whether the inventories are correct. This approach using
indicators to scale national emissions is a useful first tool for inventory verification.
Where inventory data are based on direct emission measurements, the related activity data can beused to estimate emission factors. These can be compared with the default values and any large
differences would need to be explained. Similarly, where Parties have used complex methodologies
for emissions estimation, implied emission factors, estimated by taking the inventory estimates anddividing them by a suitable simple measure of activity data, can provide useful values for comparison
with default emission factors. These systems of checks and balances can be implemented at anational level.

Atmospheric Verification
To reach the ultimate objective of the UNFCCC, the compilation of national inventories must relate
to the actual emissions to and uptakes from the atmosphere and these emissions and uptakes should
be observable as changes in the atmospheric concentrations of these gases. There are a number oftechniques using atmospheric concentrations, meteorological observations and chemical transportmodels to determine the emissions and uptakes of greenhouse gases. These techniques are applicable
on differing scales from single large sources to cities, regions, hemispheres and the globe. Thesemethods include eddy correlation, multiple tracer techniques, chemical mass balance, horizontal flux
divergence measurements and inverse modelling. Studies of this type can provide independent
verification of emissions and uptakes in national inventories.

165. How can uncertainties be evaluated and communicated?
To help Parties identify options for reducing uncertainties, the Inventories Programme will need toconsider the approaches which are available. The IPCC Revised 1996 Guidelines make use ofqualitative indicators (High, Medium, Low) and also contain default quantitative data on
uncertainties. Semi-quantitative methods have been used4, as well as detailed quantitative analyses.
The evaluation and reporting of uncertainty is central to the task of reducing uncertainty. The resultsof this evaluation can be used to help focus national efforts for improving inventory quality. The
evaluation and reporting of uncertainty is also part of good practice in inventory preparation. The
two areas are closely related and can benefit from each other. The objectives of uncertainty analyses
include producing an overall uncertainty for the inventory and allowing the comparison of
uncertainties from different sectors in the inventory and from different Parties’ inventories. To meetthese objectives uncertainties should be quantitatively determined. Because of the nature of the data
used to calculate inventories, expert judgement will often be required to perform quantitative
uncertainty analysis, and in aspects such as the choice of measurement techniques to determine
emission factors. To begin defining an approach to managing uncertainty, the Tiers or levels ofcomplexity which are available for evaluating uncertainty in the context of the IPCC inventory
guidelines need to be specified.
The group of experts at the Paris meeting first considered the development of a framework forevaluating and communicating uncertainty in national greenhouse gas inventories. An integral partof this framework was the development of a common terminology in the form of a Glossary of Terms
containing statistical definitions and technical terms related to inventories, and attempt to reconcile
the usage. The group went on to assess the range of approaches that could be used for estimating
uncertainty in emission estimates. The following section presents some of the findings of thisassessment. The Glossary as it stands at present can be found in Annex 1.
The lowest Tier of a framework for evaluating and communicating uncertainty should use a set ofdefault uncertainty factors (expressed as a percentage of the emission or uptake rate) for each subsector or class of the inventory, and the use of a default numerical method to combine the resulting
uncertainties into an overall uncertainty for the inventory.
The highest Tier, or most complex method, could use a fully quantitative assessment of theuncertainty, including the determination of statistical parameters (mean, probability distribution etc.)
for each emission factor and activity as well as an appropriate method of combining uncertainties todetermine the total uncertainty for a sector of the inventory. This highest Tier could include
verification tools such as atmospheric measurements and inverse modelling, peer review of theinventory and advanced quantitative analysis.
Most Parties will probably use a range of methods, from simple default methods to the mostcomplex, and these methods may not be the same among sectors. Because of this, it is important thatthe sector workshops on good practice provide information on the extent of knowledge about
uncertainty in the sector. This includes whatever information is available on probability distributionsof emission factors and activities and if possible parameters such as covariance of emissions
estimates, systematic errors and the extent to which quantitative uncertainty can be evaluated at thesector level. From this type of information, the approach for combining these uncertainties can be
determined.
Finally it is important to consider the extent to which peer and public review as well as quality
assurance/quality control are useful in the assessment of uncertainty. These tools are an important
4For example DARS (Data Attribute Rating System) developed by the US to characterise attributes that arecommon to all inventory data.

17

part of good practice in inventory preparation, but can also be used to evaluate the uncertainty of theinventory or conversely establish the level of confidence in the inventory.
The assessment should consider methods of communicating the uncertainty/confidence in theinventory to a non-technical audience, including policy makers. This can be useful at aninternational level but also at a national level. It is important to consider the type of audience when
deciding how to communicate this information.

18

6. How can we minimise bias?
After reviewing the tools available for ensuring good practice in inventory preparation, experts
recommended the following approach to developing guidance on this subject for inventory
practitioners.
Since inventory uncertainties will remain despite scientific advances, Parties will need to consider
how to deal with uncertain information. Parties can manage uncertainty and minimise bias byfollowing good practice guidance in compiling, assessing and reporting inventories. These goodpractice guidance should outline a high standard for inventory quality that will give confidence in theuse of inventories for assessing compliance. The good practice guidance should recognise thediffering national circumstances on data availability, resource constraints, and infrastructure, andshould recognise that inventories will improve over time. They should also recognise that thetechnological and environmental conditions that control emissions vary from country to country.
Broadly, good practice guidance should outline steps to ensure that inventories possess the following
attributes:

Completeness
Inventories should include all sources and sinks of greenhouse gas emissions, consistent with the
IPCC Revised 1996 Guidelines including sources which might be identified under the ’other’categories
Excluding sources leads directly to bias.

Use of Appropriate Methodological Approaches
The current implementation of IPCC methodologies should be reviewed and guidance developed on
the choice of appropriate estimation methods, preferably by using decision trees. Inventory reporting
should identify important sources by a methodology that considers the contribution of each source tothe size and trend of the overall inventory. Emissions factors, model parameters, activity data anduncertainty assessment for each source should reflect good practices as determined by inventory
experts in the workshops.

Inventory Quality Assessment and Review
Inventories should undergo a thorough domestic assessment from government and non-governmentalorganisations. Parties should apply a rigorous QA/QC system to their inventories to facilitate thisprocess of peer and public review. Where possible, inventory estimates should be compared withother estimates and validated by measurements.

Transparent Reporting
Inventories should meet the formal reporting requirements as established by the UNFCCC, and thereporting instructions provided by the IPCC. Good practice guidance should describe how to ensurethat there is sufficient information with which to allow a third party to reproduce the results,
including all activity data, emissions factors, model parameters, and assumptions. In addition,
inventory submissions should describe major changes in trends, methodologies, emission factors, andexplanations for any anomalies and divergence from accepted good practices. Inventories should
also describe the results of QA/QC, peer and public review, and include a transparent presentation of
uncertainty analysis.

19

Identification of Priority Areas
Inventories should call attention to areas of special concern, and to national and international efforts
to address these areas. They should encourage future scientific research and advancements in the“state of the art.”
Inventories that adhere to this good practice guidance will remain uncertain, because uncertainty isinherent in any estimating procedure. Preparing inventories in a manner consistent with the goodpractice guidance, however, should result in greater consistency and minimum bias. Those elementsof the good practice guidance that deal with transparency and reporting, moreover, should ensure thatany remaining issues related to inventory preparation or content can be clearly identified. Thus, goodpractice guidance can serve as a readily achievable means of providing Parties with more confidence
in inventory quality.

7. What further work does IPCC need to assess and co-ordinate?
The request from SBSTA8 noted that the Kyoto Protocol includes provisions related to greenhousegas inventory methods, and encourages the IPCC/OECD/IEA Inventories Programme to:•give high priority to completing its work on uncertainty•

prepare a report on good practice in inventory management•

submit a report on these issues for consideration by the SBSTA, if possible by COP5.

To meet this request, the Paris meeting agreed that IPCC should organise a series of six workshops.
Four of these will define good practice in inventory preparation by emissions sector in the context of
the IPCC 1996 Revised Guidelines. One will deal with good practice in quantifying uncertainty andrelated issues of data verification, transparency and quality control. The final workshop will produce
overall recommendations. The programme will aim to give Parties the best possible basis for theirdecisions on what inventory information is necessary in terms of choice of data and calculation
methods as well as quality control, cross checks and other institutional approaches in order for themto demonstrate compliance under the Protocol.
At the sector workshops, inventory experts will consider the good practice elements described abovein terms of how they can be applied to the development of emissions inventories at the sector andsource level. The workshops will develop recommendations on the specific components of goodpractice guidance, including: a process to identify appropriate methods using decision trees, adescription of good practice in data collection and emission factor selection and development, and a
description of good practice in inventory calculations; taking into account the recommendations fromthe uncertainties framework. Detailed information on the sector specific workshops can be found in
Annexes 2 and 3.
A workshop on approaches for estimating and reporting uncertainty and cross cutting issues such as
QA/QC will be held following the sector-specific work. This workshop will consider thedevelopment of a multi-tiered methodology for the assessment of uncertainty including definitions oftechnical terms, a default method and one or more higher Tiers which include the use of expert
judgement, appropriate measurements and a statistical methodology (including probability
distributions for emission factors and activity data) for combining these uncertainties. Thismethodology should also include guidance for communicating these uncertainties to a non-technicalaudience. Detailed information on the work of the Breakout group on the Uncertainty Frameworkand the workshop on uncertainty issues can be found in Annex 4.

20

Annexes
1. Glossary of Terms
2. Good Practice in Inventory Preparation
3. Issues for Sectoral Workshops to Consider
4. Description of the Uncertainty Framework
5. List of Papers Presented at the Meeting
6. List of Participants21