Annex 4: Description of the Uncertainty Framework
Introduction
The UNFCCC is starting probably the largest environmental monitoring programme in the world.
The uncertainty work must focus on key processes, it must consider internal and external
uncertainties in inventories and how to treat them. There is a need to properly quantify uncertainties
in order to prioritise environmental research to help provide more accurate emission estimates. The
measures of uncertainty should over time chart the improvements made in the inventory
methodology and data. Examples of this interactive activity are given in the country studies.
Uncertainty in greenhouse gas inventories is inevitable and stems from limitations to our knowledge
about the processes which generate emissions and uptakes. Key limitations include:•

our knowledge about the variety of processes which generate emissions;•the number of measurements available to develop models or algorithms for the
calculation of emissions and uptakes,•

the resources we are able to assign to collecting statistical data necessary for the
preparation of inventories, and•the inherent variability of the systems being considered.

Uncertainty can be reduced through additional research and/or expanded measurement programs, but
it cannot be completely eliminated. Therefore, we must decide how best to work with the data andresources available to us, taking full account of the need to accommodate the circumstances of all
Parties.
The national case studies presented in Paris and the assessments by the UNFCCC secretariat suggestthat the absolute uncertainty in national inventories in a given year may be 20% or more at the 95%
confidence limit. They also suggest that the uncertainty in the trend, though less than this, is still
likely to be comparable to the level of commitments.
Parties can manage uncertainty and minimise bias by following good practice guidance in compiling,assessing and reporting inventories using the IPCC yardsticks of consistency, comparability andtransparency.
As greenhouse gas inventories continue to improve, so will the ability to model atmosphericconcentrations and their trends.
When these calculated emissions agree with observed
concentrations, we will be able to effectively assess the impact of future and different emission
controls on atmospheric concentrations of greenhouse gases.

Definitions and Constraints
An essential aspect of the development of an uncertainty framework for national greenhouse gas
inventories is the development of an agreed set of definitions of the technical terms used. Definitions
can be drawn from the statistical literature, but in most cases these definitions are broader than themeanings used in the context of inventory uncertainties, and in some cases multiple meanings exist in
the statistical literature. Therefore one task of the uncertainty sub-group is to develop a glossary of
key terms incorporating both the relevant accepted statistical definition and the specific meaning of36

the word within the context of IPCC inventory uncertainty work. The first draft of this glossary is
attached to this meeting report and there is ongoing work on the Glossary. As the uncertainty
framework is fleshed out, so probably will the glossary grow.
The Glossary has the potential to provide substantial help towards the achievement of the IPCC goalsof consistency, comparability and transparency by systematising the language of reporting
uncertainties in inventories.
Some constraints concerning the use of statistical approaches can be taken from the Revised 1996
Guidelines. One specific example is that the Guidelines stipulate that the uncertainty be expressed as
2 standard deviations assuming a normal frequency distribution or approximately the 95 percent
confidence range. The 95 percent confidence range is subsequently adopted for use here as it is moregenerally applicable than the 2 standard deviations estimate. The period and areal extent ofemissions and uptakes are confined by the IPCC guidelines to apply to one year and to the land
distribution of a specific sector or sub-sector of a whole nation or Party to the Convention.
It was resolved that because of the need to have consistent and comparable inventories, and for thesequalities to apply at the sector and sub-sector levels across the inventory, that all uncertainty
estimates must be quantitative, because quantitative and qualitative uncertainty estimates can neither
be compared nor combined. There was no agreement on the proposition that quantitative information
can be derived from qualitative uncertainty estimates. Hence, consistent with the IPCC default
methodology for emissions estimates, default quantitative uncertainties estimates are required. (Ofcourse there needs to be an assessment of how these uncertainties have been derived.) The
determination of uncertainties should be done on a detailed level (each gas and each sub-sector) andfor emission factors separately from activity data.

The Development of New Uncertainty Guidelines including a Tiered ApproachThis section of the report presents the initial output of the sub-group on uncertainties and definesareas for further assessment.
The evaluation and reporting of uncertainty is central to the task of reducing uncertainty. Fromprevious efforts, it is clear that a well defined approach to evaluating and communicating uncertainty
is important to overall inventory development. The group decided to begin defining this approach byspecifying the Tiers or levels of complexity which are available for evaluating uncertainty. The
objectives of uncertainty analyses include producing an overall uncertainty for the inventory; andallowing the comparison of uncertainties from different sectors in the inventory and from different
Parties inventories. The only way to meet these objectives is to have all uncertainties quantitatively
determined.
The lowest Tier or simplest method was defined as the use of a set of default uncertainty factors(expressed as a percentage of the emission or uptake rate, or emission factor or activity) for each gas
and each sub-sector or class of the inventory, and the use of a default numerical method to combine
the resulting uncertainties into an overall uncertainty for the inventory of a Party to the Convention.
The highest Tier or most complex method was defined as a full quantitative assessment of theuncertainty, including the determination of statistical parameters (mean, probability distribution etc.)
for each emission factor and activity as well as an appropriate method of combining uncertainties todetermine the total uncertainty for either a sector or all of the inventory. Most Parties will probably
use a range of Tiers, from simple default methods to the most complex for different parts of theirinventory.
QA/QC procedures including independent auditing and good practice are a necessary part ofinventory development. QA/QC procedures, such as those embodied in the ISO9000 series or
described in guidance documents for other types of emission inventories, provide standards for37

documentation and external audit so that calculations can be checked. This ensures that each number
used in the inventory is traceable to its source. This is a valuable process which ensurestransparency, although it does not check that the best numbers and factors are being chosen. Somecountry inventory processes follow international standards such as ISO9000 (e.g. the United
Kingdom and Netherlands); others are working to achieve this. QA/QC is necessary in all Tiers, andit will add extra tasks to inventory preparation.
The basic information required for uncertainty analyses are the uncertainties of each of thecomponent parts, the activities and the emission factors. Many, perhaps most of the uncertainties
required for a full inventory have not been determined in most countries. There are common
methods of determining uncertainty used in physical sciences (determining precision by repeatedmeasurement) that can be used to determine some of these uncertainties. However there are somecases where these methods are not applicable, particularly where an activity is determined by only
one method and only once per inventory period. There is need for development and explanation of
the procedures used to determine uncertainty in such ‘single collection’ statistics. Depending on theimportance of these sources, specific programmes could be developed to improve the activity
statistics. Covariance data is another special case. In some cases expert judgement may be used todetermine uncertainties. In this case there are special needs regarding reporting and transparency, sothat the process is reproducible, if not the judgement.
There are numerical statistical techniques for combining quantitative uncertainties such as are needed
for the inventory work. One task is to test these (Monte Carlo, Latin Hypercube) with real inventory
data to build up a body of experience to identify the strengths and weaknesses of preferredtechniques. There are special features of some inventory statistics that these methods will need toallow for, including covariance of emissions estimates and systematic errors.
Full uncertainty analysis performed by Parties would lead to better understanding of the relationshipbetween uncertainties and monitoring commitments. At this stage the possibility of improvements ininventory data is the most important reason for doing an uncertainty analysis. Consequently efforts
should be focused on major emitting sectors. Uncertainty analysis also gives guidance in theconfidence in data which is relevant for the question of compliance. Uncertainty analysis must,however, be supplemented with sensitivity analysis and strict routines for quality reporting in order
to be meaningful.
The trend in emissions between a base year (e.g., 1990) and a target year (e.g. 2010) is critical forassessing how countries are meeting targets, both under the Convention and under the Protocol. So
there is a need for a rigorous understanding about how uncertainties in the emissions of different
gases and sectors vary over time. As part of this work: methodologies for the analysis of
uncertainties in trends should be developed both for the low Tier and the high Tier approach. The
special circumstances in non-Annex 1 countries need to be considered
Inventory comparisons have a very important and currently under-used role the evaluation of
uncertainties. Inventory totals can be compared to data sets assembled by international organisations,for example, the International Energy Agency, the Carbon Dioxide Information Analysis Center andthe World Resources Institute, which have estimates for CO2 and CH4 and the Global Emissions
Inventory Activity (GEIA) of the International Global Atmospheric Chemistry Project which has
spatially explicit emissions inventories for a number of key atmospheric pollutants.
To aid in the review of inventories and the identification of areas for more detailed analysis, indirect
methods of verification can be used. This is done by comparing emissions per capita, per unit GDPor per other suitable statistics across countries. Differences should be explicable in terms of
economic structure; e.g. countries with large energy contributions from hydroelectricity or nuclear
power will have lower emissions per capita. This type of analysis shows if the inventories arebroadly consistent with the socio-economic structure of the Party; it does not show that they are38

correct. This approach using indicators to scale national emissions is an ideal first tool for inventory
verification.
The ultimate objective of the UNFCCC is to stabilise “the greenhouse gas concentrations in theatmosphere…..”. Thus this commitment under convention, the compilation of national inventories,
must relate to the actual emissions to the atmosphere and should be observable in terms of
perturbations to the atmospheric concentrations of these gases. There are a number of techniques
using atmospheric concentration observations, meteorological observations and chemical transportmodelling that can be used to determine the emissions and uptakes of greenhouse gases. Use of thesetechniques can provide independent verification of emissions and uptake in national inventories.
Lastly, it is important to clearly communicate our level of confidence in the emissions data. This in
itself will contribute to better management of uncertainty. This includes having agreed definitions oftechnical terms and an understanding of the importance of uncertainties to policy makers incomparison with the perception of bias or lack of bias in the inventory calculation. A goodmechanism for communicating the confidence of the data in plain language will contribute to a better
understanding of greenhouse gas inventories. Peer and public review can be important tools in thisprocess.
Despite scientific effort, it is unlikely that this uncertainty in inventories will be substantially reducedin the near future. Therefore increased transparency in reporting is essential to ensure that all parties
can have confidence that the results produced by the emission inventories result in greater
consistency and less bias.

Input to sector workshops
The uncertainty sub-group formulated a series of questions/tasks for the sector workshops which it
thinks will help progress the work on uncertainties and good practice. These questions/tasks are:• identify default uncertainties (95% confidence limits) for any activities, emission factors andemission rates that are known in your sector. The default uncertainty is the reasonable estimate ofuncertainty to be applied to an activity, emission factor or emission rate for the national inventoryof a Party to the Convention where no local information (in that nation or Party) on theuncertainty exists;• list documentation of the source of these uncertainties;• identify sectors or sub-sectors where this cannot be done;• identify sub-sectors where participants could identify the uncertainties in their own national
inventories;• indicate what decision tree is possible for documentation purposes when deciding about the use ofdefault uncertainties;• what covariance, if any, exists between activities and emission factors that complicates thecalculation of emissions in this sector;• are there any weak spots in the methodologies, activities and emission factors for this sector ;• what documentation would you, as experts, consider necessary for transparency in this sector ;• what caveats are appropriate to emissions and uptake estimates for this sector;• how should high GWP gases and their reporting be addressed;39• do we need regional default emission factors and activity data for this sector? Do any alreadyexist?
• what documentation already exists for transparency in this sector.

The uncertainties sub-group proposed that they work with the organisers of each sector workshop
to clarify these questions/tasks and review with the organisers the results after the workshop in
preparation for the uncertainty workshop and the final synthesis workshop.

Proposed Work Program of the Uncertainties sub-group
To meet the SBSTA request concerning both good practice and uncertainties, the Paris meeting
agreed that IPCC should organise a series of six workshops. One workshop will deal withoverarching issues in quantifying, communicating and managing uncertainty and related issues ofverification, transparency and quality control. The final workshop in the series will synthesise theoutputs of the previous five workshops to produce overall recommendations. The programme will
aim to give Parties the best possible basis for their decisions on what inventory information isnecessary in terms of choice of data and calculation methods as well as quality control, cross checks
and other institutional approaches in order for them to demonstrate compliance under the Protocol.
The workshop on approaches for estimating and reporting uncertainty will consider the developmentof a multi-tiered methodology for the assessment of uncertainty including definitions of technicalterms, a default method and one or more higher Tiers which include the use of expert judgement,
appropriate measurements and a statistical methodology (including probability distributions foremission factors and activity data) for combining these uncertainties. This methodology should also
include guidance for quality assurance/quality control and for communicating these uncertainties to a
non-technical audience.
There will be a e-mail working group set up to progress the issues in advance of the uncertainties
workshop. The research that is necessary to develop a systematic approach to uncertainties ininventories includes:• the selection of appropriate statistical methods that can be used to calculate the uncertainty ofemissions data;• determination of the input data required by these methodologies (e.g. probability distributions,covariances);• development of procedures for the determination of these input data, including measurement andstatistical techniques and expert judgement;• methods of auditing and requirements for transparency for uncertainty data in particular tominimise bias;• the use of uncertainty data as indicators of inventory quality and to prioritise possible new workfor the improvement of emissions estimates;• methodologies for determination of uncertainties in emission trends ;• a Tiered system that combines these features into a manageable Uncertainties system for the IPCC
Inventories Guidelines;• evaluation of inventory uncertainties from inventory comparisons both within IPCC and outside;40• an assessment of the capability of atmospheric observations to provide independent confirmationof inventory estimates; and• guidance on the overarching issues with regard to managing and communicating uncertainties ininventories.

Preliminary work addressing the various tasks will be performed by an e-mail discussion group on
uncertainties hosted by the IPCC/OECD/IEA Inventories Unit. The uncertainties workshop is
expected to take place in Oct 1999. Members of the uncertainty group will provide input to thesector good practice workshops and take feedback from these workshops.

Suggested Agenda for Uncertainties Group
1. The uncertainties electronic discussion group be set up
2. The first task of the group should be to prepare a refined glossary acceptable to those working in
the area, containing for each term (a) an appropriate definition from the statistical literature and(b) the more specific meaning of the term when used in the context of IPCC National Inventories.
The logic of setting this task first is that the group will start off by checking that it is speaking acommon language.
3. That a parallel task be to provide input and feedback through the chairs and the Secretariat, to thesector good practice workshops.
4. That the group prepare a first draft of a document for the IPCC overall Guidelines on
Uncertainties, of which the proposed contents list is below. That in the first instance as part ofthis methodology, only two tiers be developed, a default set of emission factor and activity
uncertainties, tier 1, and a procedure for determining individual uncertainties by measurement,
expert judgement etc., tier 2.
5. That the contents of the draft IPCC overall Guidelines on Uncertainties include:a) Introduction (as exists)b) Definitions and Constraints (an introduction to the glossary etc.)c) Quality Control/Quality Assurance (how to ensure transparency, comparability, consistency,
completeness, the theory and practice of QA/QC with regard to IPCC national inventories,
information concerning accepted auditing standards e.g. ISO9000)d) Uncertainty Analysis (introduction, description of appropriate statistical methods that can beused to calculate the uncertainty of emissions data, description of practical approaches,particularly commercially available software for these calculations, determination of the
input data required by these methodologies, e.g. probability distributions, covariances)e) Determining the Uncertainties in the Input Data (procedures for the determination of
uncertainties in input data particularly activity data and emission factors, default values fortier 1, measurements and statistical techniques and expert judgement for tier 2)f) Worked examples (showing how the methods work)g) Inventory Comparisons as Uncertainty Analyses (guidance on how to evaluate inventory
uncertainties from inventory comparisons both within other national IPCC inventories andinventories from outside sources, and from atmospheric observations that provide
independent assessments of inventory emissions and uptakes, note not how to make suchmeasurements just guidance on the comparisons)41h) Inventory Development Research Priorities (the use of uncertainty data as indicators of
inventory quality and to prioritise possible new work for the improvement of emissions
estimates)i)

Uncertainties and Trend Analyses (methodologies for determination of uncertainties inemission trends)

j)Managing and Communicating Uncertainties in Inventories (including uncertainties andbias).

k) Glossary (as discussed)
That a major task of the Uncertainties workshop be to review/revise/prepare these draft guidelines in
the light of what has been prepared and further input including that from the two previous sector
good practice workshops and to prepare the appropriate overarching information for the concluding
workshop.

42