Quality Assurance and Quality Control

5.5

QUALITY ASSURANCE AND QUALITYCONTROL

5.5.1

Introduction

The IPCC Good Practice Guidance and Uncertainty Management (GPG2000, IPCC, 2000), Chapter 8, Quality
Assurance and Quality Control, defines quality assurance (QA) and quality control (QC), and provides guidanceon the elements of a QA/QC system, taking into account the need for transparency and review. It also discusses
the practical issues that inventory agencies must consider when allocating resources to QA/QC across the entire
inventory and how to rationalise the prioritisation resources for the LULUCF sector. This section enumerates thetypes of procedures that an inventory agency should undertake in order to ensure that the inventory estimates andtheir contributing data are of high quality, with particular emphasis on issues in the LULUCF sector. The
procedures also contribute to developing an inventory that can be readily assessed in terms of quality andcompleteness.
BOX 5.5.1
DEFINITIONS OF QUALITY ASSURANCE AND QUALITY CONTROL

Quality Control (QC) is a system of routine technical activities, to measure and control the qualityof the inventory as it is being developed. The QC system is designed to:(i) provide routine and consistent checks to ensure data integrity, correctness, and completeness;(ii) identify and address errors and omissions;(iii) document and archive inventory material and record all QC activities.
QC activities include general methods such as accuracy checks on data acquisition andcalculations and the use of approved standardised procedures for emission calculations,
measurements, estimating uncertainties, archiving information and reporting. Higher tier QC
activities include technical reviews of source or sink categories, activity and emission factor data,
and methods.
Quality Assurance (QA) activities include a planned system of review procedures conducted bypersonnel not directly involved in the inventory compilation/development process. Reviews,
preferably by independent third parties, should be performed upon a finalised inventory following
the implementation of QC procedures. Reviews verify that data quality objectives were met,
ensure that the inventory represents the best possible estimates of emissions and sinks given thecurrent state of scientific knowledge and data available, and support the effectiveness of the QC
programme.
Source: IPCC (2000).
Box 5.5.1 presents the definitions of quality control and quality assurance used in GPG2000. GPG2000 also
identified the following elements of a complete QA/QC system:•

An inventory agency responsible for coordinating QA/QC activities;•

A QA/QC plan;•

General QC procedures (Tier 1) that cross-cut all inventory categories;•

Source or sink category-specific QC procedures (Tier 2) that require knowledge of data and methods;•

QA review procedures;•

Reporting, documentation, and archiving procedures.

The inventory methods for the LULUCF sector require specific good practice guidance for QA/QC in all but thefirst of these elements. In addition, verification issues and issues related to the Kyoto Protocol can affect QA/QC
good practice. These two issues are addressed in Sections 5.7 and 5.5.7, respectively.
Estimating emissions and removals from LULUCF activities involves several important – although notnecessarily unique – issues. The primary difference between the LUCF sector and other sectors in the IPCC
Guidelines (IPCC, 1997) (i.e., energy, agriculture) is that the LUCF sector focuses on calculating the net

IPCC Good Practice Guidance for LULUCF

5.49

Chapter 5: Cross- Cutting Issues

emissions or removals.17 In particular, the QA/QC system must recognise that the LULUCF sector is unique
because CO2 can be both removed from and emitted to the atmosphere. From the perspective of inventory
QA/QC, however, more important considerations in the LULUCF sector focus on the complexity of the data thatare needed for preparation of accurate estimates of emissions and removals from LULUCF. Four importantfeatures of LULUCF inventory methods that generally affect QA/QC are highlighted below.
•

Representativeness of input data: LULUCF activities affect large geographical areas. Because of the sizeof these areas – coupled with the complex nature of the biological processes taking place – it is impracticalto rely entirely on direct measurements of greenhouse gas emissions and removals in producing national
inventories. Consequently, inventories rely on data produced using sampling through field measurements
and land surveys. Further, a complete set of samples is not likely to be taken on an annual basis, but instead
will be taken periodically (e.g., every four years). Samples may also be augmented with remote sensing datathat allow more complete coverage.

•

Need for historical data: Greenhouse gas emissions and removals related to LULUCF is a function of pastland-use activities, which continue to affect current (i.e., inventory year) CO2 emissions or removals. Thus,
both past and current land use and forestry activities influence current emissions and removals. For thisreason, sufficient historical data are needed to assess present day emissions, and so the datasets used in the
LULUCF sector may cover a longer historical period than other source categories (e.g., 20 to 100 years).
However, many countries benefit from the fact that forestry and some other land-use data have beencollected for a long time, so detailed and comprehensive – although not necessarily accurate – data sources
may be available.18 Time series consistency is an important QA/QC issue and is discussed in more detail in
Section 5.6.

•

Complex interactions and variability of the biological processes: The complex interactions and inherent
variability of the biological processes associated with forests, soils, and other LULUCF components canlead to the need for use of more sophisticated models19 than those employed for estimating emissions frommost other source categories. The data, assumptions, and other characteristics of the model may not always
be transparent. QA/QC needs to focus on documenting model characteristics and assumptions, checking
model outputs, identifying areas for improvement, checking the model algorithms, and documenting theresults of those checks.

•

Variability in the magnitude and nature of the data: Greenhouse gas emissions or removals can be small
net fluxes resulting from large gross fluxes or differences between large stocks, for example slow changes inlarge soil organic carbon stocks in soils. In addition, different types of activities lead to different types ofchanges. For example, forest management is likely to result in small and dispersed changes per unit area
over large areas, whereas large scale deforestation results in relatively large and immediate net emissions.
For these reasons, QA/QC procedures should involve the assessment of the suitability of the selected
methods for estimation of the greenhouse gas in each case, from direct measurements to sophisticatedmodels.20

5.5.2

QA/QC Plan

As discussed in GPG2000, a QA/QC plan is a fundamental element of a QA/QC system, and it is good practice
to develop one. The plan should, in general, outline the QA/QC activities that will be implemented, and include ascheduled time frame that follows inventory preparation from its initial development through to final reporting in
any year. It should also contain an outline of the processes and schedule to review all source and sink categories.
For LULUCF source and sink categories, the plan should describe the specific QC procedures that have been or
will be implemented in addition to special QA review procedures employed. These procedures should be

17

It should be noted, however, that subtracting major components during an emission source category calculation, is notunique to LULUCF sector. For example, thoroughly estimating carbon storage in non-energy fossil fuel feedstocks
involves a complicated analysis of fossil fuel processing and fates in order to subtract the amount of carbon in those fuelsthat is not combusted or oxidized. These adjustments to fossil fuel combustion calculations can be quite significant relative
to a country’s overall emissions inventory.

18

Of course these data will have been collected for reasons other than estimating greenhouse gas emissions and removals.

19

Numerical or process models interpolate activity data for intermediate years between samples, extrapolate sample datafrom measures of timber volume or other metrics to total biomass carbon, and attempt to capture other complexities andsubtleties of the relationship of forestry and land- use change to emissions and removals of CO2 and other gases.

20

The issue of methodological choice is discussed in detail at the subcategory level in Chapter 3 of this report.

5.50

IPCC Good Practice Guidance for LULUCF

Quality Assurance and Quality Control

formulated in such a way that they address the four features described in Section 5.5.1, the representation of land
areas in Chapter 2 (Basis for Consistent Representation of Land Areas), LULUCF sector methodologies in
Chapter 3 (LUCF Sector Good Practice Guidance), and, if relevant, the methods used for accounting emissions
and removals under Article 3.3 and 3.4 of Kyoto Protocol in Chapter 4 (Supplementary Methods and Good
Practice Guidance Arising form the Kyoto Protocol).

5.5.3

General QC Procedures (Tier 1)

It is good practice to implement the generic QC checks as outlined in GPG2000, Chapter 8 (Quality Assurance
and Quality Control) Tier 1 General Inventory Level QC Procedures. These general techniques focus on theprocessing, handling, documenting, archiving, and reporting procedures that should be used for all inventory
source and sink categories. Table 5.5.1 lists the generic Tier 1 QC checks from Table 8.1 in GPG2000. Thesechecks have been revised to make them applicable to sinks as well as sources. In cases where estimates for the
LULUCF sector are prepared by institutions other than the inventory agency, the inventory agency is still
responsible for ensuring that Tier 1 QC procedures are performed and that both findings and procedures are
documented.
TABLE 5.5.1
TIER 1 GENERAL INVENTORY LEVEL QC PROCEDURES
QC Activity
Check that assumptions andcriteria for the selection ofactivity data, emission
factors and other estimation
parameters are documented.
Check for transcription
errors in data input andreference.
Check that emissions andremovals are calculated
correctly.
Check that parameter andunits are correctly recorded
and that appropriate
conversion factors are used.

Check the integrity of
database files.

Check for consistency indata between categories.
Check that the movement of
inventory data amongprocessing steps is correct.

Check that uncertainties inemissions and removals are
estimated or calculated
correctly.

Procedures•

Cross- check descriptions of activity data, emission factors and other estimation
parameters with information on source and sink categories and ensure that these areproperly recorded and archived.

•

Confirm that bibliographical data references are properly cited in the internal
documentation.

•

Cross- check a sample of input data from each source category (either measurementsor parameters used in calculations) for transcription errors.

•

Reproduce a representative sample of emission or removal calculations.

•

Selectively mimic complex model calculations with abbreviated calculations to judge
relative accuracy.

•

Check that units are properly labelled in calculation sheets.

•

Check that units are correctly carried through from beginning to end of calculations.

•

Check that conversion factors are correct.

•

Check that temporal and spatial adjustment factors are used correctly.

•

Confirm that the appropriate data processing steps are correctly represented in the
database.

•

Confirm that data relationships are correctly represented in the database.

•

Ensure that data fields are properly labelled and have the correct design specifications.

•

Ensure that adequate documentation of database and model structure and operation arearchived.

•

Identify parameters (e.g., activity data, and constants) that are common to multiple
categories of sources and sinks, and confirm that there is consistency in the values
used for these parameters in the emissions calculations.

•

Check that emission and removal data are correctly aggregated from lower reportinglevels to higher reporting levels when preparing summaries.

•

Check that emission and removal data are correctly transcribed between different
intermediate products.

•

Check that qualifications of individuals providing expert judgement for uncertainty
estimates are appropriate.

•

Check that qualifications, assumptions and expert judgements are recorded. Check that
calculated uncertainties are complete and calculated correctly.

•

If necessary, duplicate error calculations on a small sample of the probability
distributions used by Monte Carlo analyses.

IPCC Good Practice Guidance for LULUCF

5.51

Chapter 5: Cross- Cutting Issues

TABLE 5.5.1 (CONTINUED)TIER 1 GENERAL INVENTORY LEVEL QC PROCEDURES

Undertake review ofinternal documentation.

Check time seriesconsistency.

Undertake completenesschecks.

Compare estimates toprevious estimates.

5.5.4

•

Check that there is detailed internal documentation to support the estimates and enablereproduction of the emission and removal and uncertainty estimates.

•

Check that inventory data, supporting data, and inventory records are archived andstored to facilitate detailed review.

•

Check integrity of any data archiving arrangements of outside organisations involved
in inventory preparation.

•

Check for temporal consistency in time series input data for each category of sources
and sinks.

•

Check for consistency in the algorithm/method used for calculations throughout thetime series.

•

Check recalculation method.

•

Confirm that estimates are reported for all categories of sources and sinks and for all
years from the appropriate base year to the period of the current inventory.

•

Check that known data gaps that result in incomplete emissions estimates are
documented.

•

For each category, current inventory estimates should be compared to previous
estimates, if available. If there are significant changes or departures from expected
trends, re- check estimates and explain any difference.

Source or Sink Category-Specific QC Procedures(Tier 2)

It is good practice to supplement the Tier 1 QC checks related to data processing, handling and reporting with
Tier 2 source or sink category-specific procedures for key categories (i.e., with the additional quality control
checks outlined in GPG2000, Section 8.7, Source Category-Specific QC Procedures (Tier 2)). Tier 2 proceduresshould be implemented on a case-by-case basis. These checks may be applicable, particularly if higher tierinventory methods are used to prepare emission and removal estimates. The Tier 2 QC procedures are directed at
specific types of data used in the methods and require knowledge of the source or sink category, the types of data
available, and the parameters associated with emissions or removals.
In some cases, the quantity and complexity of data that will be used to develop estimates of emissions andremovals from LULUCF may lead to some difficulties for implementing Tier 2 QC checks and investigations. At
the same time, this complexity makes it all the more important that rigorous Tier 2 data quality investigations be
performed and that they be done in cooperation with the institutions that are primarily responsible for collecting
and analyzing LULUCF data. These institutions may be numerous and somewhat diverse because of theallocation of land management responsibilities within each country. Investigating the quality of the input data
used in LULUCF models and other calculations will require extensive cooperation and communication withthese institutions to better understand their existing QA/QC procedures.
While source and sink category-specific checks are described in Chapter 3 of this report, Tier 2 QC for the
LULUCF sector should focus on the following types of checks:•

The inventory agency should check that land areas are properly classified and that no double counting oromissions of land area have occurred (see Section 2.3.2 of Chapter 2 and Table 2.3.1) This land areaclassification should be consistent with Chapter 2 (Basis for Consistent Representation of Land Areas). In
particular, it is important to check consistency and possible double-counting between the agriculture sector
and the LULUCF sector.

•

The inventory agency should investigate the completeness of source and sink categories in the LULUCF
sector, by examining the land-use categories and the subcategories to the extent appropriate, as described in
Chapter 3 (see Table 3.1.1 and Table 3.1.2 in Section 3.1.1). This is particularly important because of thecomplicated relationships among several of the LULUCF categories (e.g., abandoned lands regrowing andchanges in woody biomass stocks) and between LULUCF categories and other source categories (e.g.,biomass cleared and biomass fuel combustion). This classification should be consistent with Chapter 3,(LUCF Sector Good Practice Guidance). The inventory agency should also assess whether estimates ofparticular categories cover all relevant geographical areas (e.g., territories), sub-source or sink categories,
pools, or activities.

5.52

IPCC Good Practice Guidance for LULUCF

Quality Assurance and Quality Control•

The inventory agency should periodically check the consistency of the time-series activity data, because of
the long history of data needed to estimate emissions for a single year. The activity and other data usedshould represent a consistent land area for the country, and have been collected using methods which do not
introduce temporal biases. Discontinuities in the time series of emissions or other data used in thecalculation of emissions or removals should be explained. The direction and magnitude of theemissions/removals estimates for individual LULUCF source or sink categories and their subcategories
should be compared and assessed as to the reasonableness and causes of these changes, considering thepossible impact of climate variability on time scales (for example at the scales of decades).

•Because of the relative importance of sampling data for preparing estimates, the inventory agency should
examine the sampling and extrapolation protocols that have been used, determine what review the protocols
have undergone, identify any internal QA/QC procedures that were in place, and consider other relevant
factors. See also Section 5.3, Sampling of this report. Additional information on secondary data
investigations can be found in Section 8.7.2.1, National Level Activity Data, of Chapter 8 of GPG2000.

•

Because the multiple uses of remote sensing techniques and data for preparing the LULUCF inventory, theinventory agency should provide documentation about the data and tools being used (i.e., type of imagery
and processing) at the level of detail needed for each case.

•

Models can be a necessary part of the national inventory process. They provide the opportunity to create
regional or national estimations when scientific knowledge or available information is limited to specificlocations or conditions. Because models are a means of extrapolating and/or interpolating what one knowsin order to estimate what one is less sure of, simply assuming that the model chosen is providing accurate
output for the inventory needs to be carefully avoided. If QA/QC associated with models is inadequate or
not transparent, the inventory agency should attempt to establish checks on the models and data. In
particular, the inventory agency should check the following:(i)Appropriateness of model assumptions, extrapolations, interpolations, calibration-based
modifications, data characteristics, and their applicability to the greenhouse gas inventory method
and national circumstances;(ii)

Availability of model documentation, including descriptions, assumptions, rationale, and scientificevidence and references supporting the approach and parameters used to model land-use processes;(iii)

Types of QA/QC procedures performed by model developers and data suppliers and whether or nottheir quality control procedures are adequate;(iv)

Existence of plans to periodically evaluate and update or replace assumptions with appropriate newmeasurements. Key assumptions may be identified by performing sensitivity analyses.

5.5.5

QA Review ProceduresGood practice for QA procedures requires an expert review to assess the quality of the inventory, and also to
identify areas where improvements could be made. The inventory may be reviewed as a whole or in parts. QA
procedures are used in addition to Tier 1 and Tier 2 QC. The objective in QA implementation is to involve
reviewers that can conduct an unbiased review of the inventory. It is good practice to use QA reviewers thathave not been involved in preparing the inventory. Preferably, these reviewers would be independent experts
from other agencies or a national or international expert or group not closely connected with national inventory
compilation. Where third party reviewers outside the inventory agency are not available, staff from another partof the inventory agency not involved in the portion being reviewed can also fulfil QA roles.
It is good practice for inventory agencies to conduct a basic expert peer review (Tier 1 QA) prior to inventory
submission, in order to identify potential problems and make corrections where possible. It is also good practice
to apply this review to all source and sink categories and sectors in the inventory. However, this will not always
be practical due to timing and resource constraints. Key categories should be given priority, as well as categories
where significant changes in methods or data have been made. Inventory agencies may also choose to perform
more extensive peer reviews or audits or both as additional QA procedures within the available resources.
Inventory agencies should also consider applying the techniques and procedures for the LULUCF sector
described in Section 5.7, Verification, of this report, subject to the availability of data for these techniques andresource constraints. Priority should be given to key source and sink categories in the application of these more
rigorous verification techniques. The comparison of emission or removal estimates or other relevant data for the
LULUCF sector with data external to the inventory process can help to establish the reliability of individualcomponents. Verification of the inventory may be especially useful for the LULUCF sector, because of thepotentially large uncertainties surrounding the inventory estimates. Expert reviews and Tier 2 QC investigations

IPCC Good Practice Guidance for LULUCF

5.53

Chapter 5: Cross- Cutting Issues

are critical first steps in verification. Box 5.5.2 provides further discussion on conducting an expert peer review
for the LULUCF sector.
BOX 5.5.2
EXPERT PEER REVIEW

Expert peer review consists of a review of calculations or assumptions by experts in relevant
technical fields. This procedure is generally accomplished by reviewing documentation associatedwith the methods and results, but usually does not include rigorous certification of data or
references such as might be undertaken in an audit. The objective of the expert peer review is toensure that the inventory’s results, assumptions, and methods are reasonably judged by thoseknowledgeable in the specific field. Expert review processes in the LULUCF sector may involve
technical experts as well as researchers. Where a country has formal stakeholder and public review
mechanisms in place, these reviews can supplement but not replace expert peer review.
In the LULUCF sector, the complexity of models may make peer review more difficult, as well as
more important. Consequently, good practice should include:●

Identifying whether the major models used for the analysis have undergone peer review; if
not, the inventory agency should initiate a peer review process for the models separately, oras part of, the inventory peer review process.

●

Determining whether the documentation of the models, input data, and other assumptions,etc., is sufficiently thorough and sufficient to support the peer review.

There are no standard tools or mechanisms for expert peer review, and its use should be consideredon a case-by-case basis. If there is a high level of uncertainty associated with an emission or
removal estimate for a category, expert peer review may provide information to improve theestimate, or at least to better quantify the uncertainty. Effective peer reviews often involve
identifying and contacting key independent organizations or institutions, including researchorganizations. In the LULUCF sector, for example, the participation of researchers and researchorganizations is often needed when applying verification techniques and procedures (see Section
5.7), especially with regards to more complicated models. It is good practice to obtain the relevant
expertise in development and review of methods, data acquisition, and models.

5.5.6

Documentation, Archiving and Reporting

It is good practice to document and archive all information required to produce the national inventory estimatesas outlined in GPG2000 (Chapter 8, Quality Assurance and Quality Control, Section 8.10.1, Internal
Documentation and Archiving) including the results of the verification activities and changes in data inputs andmethods from previous years. To ensure transparency, documentation should be sufficient to enable theassessment of the estimates of emissions for key categories. Documentation and archiving procedures in the
LULUCF sector should be focus on the following issues:•

Because of the likely use of sample data and because annual data are unlikely to be available for areas,
stocks and estimation parameters, documentation of the consistency of time series data and methods forinterpolating between samples and years is particularly important.

•

Because of the importance of clear land-use classification in each year and accurate verifiable tracking of
categories over time, documentation should be provided on land-use categories.

•

Because of the complexity of LULUCF data and models, providing thorough documentation allows internal
QC checks and investigations and external QA reviews to operate effectively:

5.54

(i)

The rationale for the choice of models and their consistency with the good practice guidance
provided in Chapter 3 should be discussed, documented, and archived;(ii)

Archives should contain documentation provided by the model developers on the assumptions andworkings of the model, including data sources, source code (if available) and other information(such as sensitivity analyses);(iii)

Documentation should include data on QA/QC procedures governing models, both existingprocedures or documentation available from model developers, and efforts to institute additional or
expanded procedures.

IPCC Good Practice Guidance for LULUCF

Quality Assurance and Quality Control

5.5.7

Issues under Kyoto Protocol Articles 3.3 and 3.4

It is good practice to follow the Tier 1 and Tier 2 QC procedures described in Section 5.5.3 and 5.5.4 forestimates reported under Articles 3.3 and 3.4 of the Kyoto Protocol 21 . For the most part, the QA/QC
requirements for estimates of LULUCF prepared under the Kyoto Protocol will be similar to those for any otherinventory estimates, but there is a need to undertake additional checks according to Chapter 4. A summary ofthese Tier 2 QC checks is given below:•

Identify the geographical location of the boundaries of the area that encompasses land subjected to theactivities under Articles 3.3 and 3.4 (if elected). Special care is needed for Kyoto Protocol reporting on theattribution of specific activities to relevant land categories in tracking the shifts of an area of land from one
category to another, when different activities are taking place, one after the other, within or between
commitment periods under the Kyoto Protocol. It is also important to take into account the special
requirements for methodological choice as explained in Chapter 4.

•

Check availability of data for estimation of net-net accounting for some activities under Article 3.4 of the
Kyoto Protocol. It is important to document estimates both for the base year and commitment period. It isparticularly important to document any approximations required to estimate data for the base year.

•

Ensure that the historical data undergo QC checks that are as rigorous as the current year data.

•

Check the analysis conducted to determine that a pool which is not being reported is not a source.

21

The present section only deals with activities specified in Article 3.3 and 3.4 under Kyoto Protocol of the United Nations
Framework Convention on Climate Change (UNFCCC), it does not address projects (under Article 6 or 12 of Kyoto
Protocol).

IPCC Good Practice Guidance for LULUCF

5.55