A Guide to the Integration ofConsumer Price Index (CPI) andInternational Comparison Program (ICP)
Production Activities

2021

A Guide to the Integration ofConsumer Price Index (CPI)andInternational Comparison Program (ICP)
Production Activities1

July 14, 2021

1

This guide was prepared by Patrick Kelly, Statistics South Africa, and D.S. Prasada Rao, theUniversity of Queensland, Australia, with case study contributions from Dilip Kumar Sinha, Ministryof Statistics and Program Implementation, India; Merav Yiftach and Yoel Finkel, Central Bureau of
Statistics, Israel; Lekau Ranoto, Statistics South Africa; and Sharne Bailey, Office for National
Statistics, United Kingdom. Nada Hamadeh, Marko Rissanen, William Vigil Oliver, Elizabeth Purdie
and Rui Costa, World Bank, provided inputs to this work. The guide was reviewed by the Inter
Secretariat Working Group on Price Statistics (IWGPS) and the International Comparison Program(ICP) Inter-Agency Coordination Group. Cover design by Will Kemp.

Contents
Abbreviations........................................................................................................................................ 3
1.

Background ................................................................................................................................... 4

2.

About this guide............................................................................................................................ 53.

CPI and ICP: Similarities and differences ...................................................................................... 6

4.

Benefits of CPI-ICP integration ..................................................................................................... 8

5.

Current level of integration of CPI-ICP activities ........................................................................10

6.

The Statistical Value Chain as a basis for CPI-ICP integration ....................................................14

7.

Scanner data and web-scraped prices ........................................................................................30

8.

Housing and dwelling services ....................................................................................................31

9.

Conclusion and next steps ..........................................................................................................31

References ..........................................................................................................................................33
Appendix A: CPI-ICP integration questionnaire ..................................................................................34
Appendix B: Case studies ....................................................................................................................38
CPI/ICP case study: India ....................................................................................................................39
CPI/ICP case study: Israel....................................................................................................................43
CPI/ICP case study: United Kingdom ..................................................................................................47
CPI/ICP case study: South Africa.........................................................................................................57

2

Abbreviations
BNR
CPD
CPI
GTIN
ICP
NSO
PPP
SAF
SDG
SPD
SVC

brand not relevant
country-product-dummy method
Consumer Price Index
Global Trade Item Number
International Comparison Program
national statistical officepurchasing power parity
spatial adjustment factor
Sustainable Development Goal
Structured Product Description
statistical value chain3

1. Background
The International Comparison Program (ICP) is one of the world’s largest statistical initiatives andone of the most enduring, having celebrated its 50th anniversary in 2018. The ICP is carried out under
the auspices of the United Nations Statistical Commission (UNSC) and managed by the ICP Global
Office at the World Bank. The regional implementing agencies coordinate the regional ICP programs
while the national implementing agencies are responsible for planning and implementing ICP
activities at the country level, including carrying out price surveys and compiling the national accountexpenditure data. These stakeholders work within the ICP governance framework,2 which aims toensure that the global, regional, and national efforts to produce reliable PPP estimates and related
measures of real expenditures adhere to approved policies, protocols, methodologies and quality
assurance standards, and that the estimates are produced efficiently in keeping with available
resources.
The results of the most recent ICP 2017 cycle covering 176 economies were released in May 2020.
The ICP 2017 cycle delivered a comprehensive set of robust and high-quality results, strengthening
data quality assurance approaches and providing a fully documented and transparent process forcomputing results at the global level, whilst maintaining methods and processes consistent with theprevious ICP 2011 cycle.
Nonetheless, the sustainability of ICP as a global statistical program remains a concern to the UNSC
and other stakeholders. At the conclusion of the forty-seventh Session in 2016, the UNSC resolvedthat the ICP be a “permanent element of the global statistical programme to be conducted at more
frequent intervals.” The Commission agreed “with the proposal to adopt a rolling survey approach3
spread over a three-year cycle, starting in 2017, which would allow flexibility in conducting thesurveys according to the specific conditions of the participating countries.”4 The UNSC has further
urged participating countries to integrate and harmonize ICP activities with their regular work
program, especially the Consumer Price Index (CPI), to ensure a sustainable and truly permanent
ICP. Furthermore, the UNSC constituted the Friends of the Chair (FOC) group to conduct anevaluation of the ICP 2011 cycle. Along with a host of recommendations, the FOC noted that the
integration of the ICP and CPI activities at the country level can lower the overall costs of the ICP, as2

http://pubdocs.worldbank.org/en/255521487200449880/ICP-GB01-Doc-Governance-Framework-Final.pdf

3

The rolling survey approach over a three-year cycle requires the adoption of a rolling price survey, whereby
the participating economies collect prices of one-third of the items included in the household consumption
product list in each year over a three-year cycle. For example, comparisons for the benchmark year 2024 would
make use of prices of one-third of the items collected in each of the years 2023, 2024 and 2025. Using theclosest related and most detailed component of the CPI, item-level prices in 2023 are extrapolated whereas
prices in 2025 are retropolated to the benchmark year, 2024. The feasibility of implementing a rolling pricesurvey approach hinges critically on the availability of reliable measures on price changes over time at adisaggregated level and that are closely aligned with the ICP framework for PPPs.
4

Names of the organizational units and technical terms have been harmonized in this document, including
direct quotes from existing reports, papers, and documents.

4

well as enhance the quality, coverage, and reliability of the generated price comparisons for thehousehold consumption components of ICP.
The ICP Technical Advisory Group (TAG) considered the FOC recommendations and, in response,
prepared a research agenda5 to guide immediate and longer term ICP activities. This agenda includes
an item to conduct research on harmonizing ICP and CPI activities,6 in order to identify synergies
between the two programs and lessen the data collection burden on economies participating in the
ICP. Subsequently, the TAG established a Task Force on Country Operations and Guidelines,7 which
was entrusted with the preparation of a guide on CPI-ICP integration.8
Concurrently, there have been developments on subnational purchasing power parities (SN-PPPs)
and a guide to the estimation of SN-PPPs has been published9 (International Comparison Program,
2021). The World Bank in its submission to the forty-seventh Session of the UNSC indicated: “Tomeet the growing demand for official estimates of PPPs at the subnational level, selected countries
in various regions have undertaken the compilation of subnational PPPs in the interim period. The
ICP Global Office, in collaboration with the regional implementing agencies, is compiling an inventoryof subnational PPP projects and exploring future expansion and harmonization of these exercises tomaximize synergies and comparability.” Recognizing the importance of subnational PPPs, the Task
Force on Country Operations and Guidelines was asked to prepare the aforementioned guide on
subnational PPPs. As subnational price comparisons complement the CPI, which is a measure of pricechanges over time, the success of any such program would rely heavily on the inter-relationships
between CPI and subnational PPP compilation.
In view of these developments and the increasing reliance of the ICP on the availability of reliable
and timely CPI measures at a disaggregated level, there is a need to examine and consider a closer
integration of CPI and ICP related activities at the national level. This guide is designed to serve thispurpose.

2. About this guide
The primary audience for this guide is producers of official statistics and, in particular, price statistics
units in national statistics offices (NSOs) who are charged with collating national CPIs and their5

http://pubdocs.worldbank.org/en/774201507663170526/pdf/ICP-GB02-Doc-ICP-Research-Agenda.pdf

6https://www.worldbank.org/en/programs/icp/brief/ra07

7

Prasada Rao, University of Queensland; Brian Graf, International Monetary Fund; David Roberts, Independent
Expert; Dilip Kumar Sinha, India Ministry of Statistics and Program Implementation; Levan Gogoberishvili,International Monetary Fund; Luigi Biggeri, University of Florence; Majed Skaini, United Nations Economic andSocial Commission for Western Asia; Patrick Kelly, National Statistical Office of South Africa; Yuri Dikhanov,World Bank; William Vigil-Oliver, World Bank.
8

http://pubdocs.worldbank.org/en/307141510262316507/pdf/ICP-TF-OGP01-Doc-TOR.pdf

9https://thedocs.worldbank.org/en/doc/5064f2288436664bc8f9811c8a5b8c55-0050022021/original/GuideSubnational-PPPs.pdf

5country’s contribution to the ICP. Price statisticians in developing countries where dedicatedresources for the sustained collection of ICP data are scarce, if not readily available, will benefit mostfrom this guide. Furthermore, the guide will provide all users of official statistics with a goodunderstanding of CPIs and the ICP and their differences and similarities.
The approach taken in this guide is to identify potential points of the CPI survey process where ICP
activities can be incorporated, and to illustrate how the ICP requirements for item identification andprice collection can be satisfied by improving and standardizing the description of CPI specifications.
However, price statisticians should be mindful of not compromising the accuracy andrepresentativity of the CPI when considering new approaches.
This guide draws on the new CPI Manual: Concepts and Methods (IMF et al, 2020, and hitherto
referred to as the “CPI Manual”), 10 published by the member agencies of the Inter Secretariat
Working Group on Price Statistics,11 and adopted by the UNSC in March 2020. The CPI Manual
includes many practical suggestions which should form the foundation for incorporating ICP
requirements into CPI processes, and Appendix 5 of the CPI Manual is dedicated to an explanationof ICP methods and a summary of the benefits of CPI and ICP integration.
The following chapters in this guide review the similarities and differences of national CPIs and the
ICP (Chapter 3); the benefits arising from efforts to integrate the two exercises (Chapter 4); thecurrent level of integration in ICP participating countries (Chapter 5); where integration can takeplace in the statistical value chain, the classification and hierarchies of products, and the use of
Structured Product Descriptions (SPDs) in identifying products to be priced in both the ICP and CPI(Chapter 6); the use of scanner data and web-scraped prices for CPIs and the ICP (Chapter 7); thetreatment of housing and dwelling services data (Chapter 8); and conclusions and next steps(Chapter 9). Furthermore, the CPI-ICP integration survey sent to countries is provided at Appendix
A, and a set of national case studies illustrating the ambitions, successes, and challenges thatcountries have encountered in pursuing the integration of CPI and ICP activities is presented inAppendix B.

3. CPI and ICP: Similarities and differences
PPPs, as calculated by the ICP, and CPIs share conceptual similarities. A national CPI measures therelative change over time in the prices of goods and services within a country, whereas the ICP’s
PPPs measure relativities in prices, denominated in local currencies, across countries at a given pointof time, and thus provide a spatial comparison. Therefore, the CPI and PPPs facilitate, respectively,
temporal and spatial measures of price change. It is also important to emphasize that national CPIs
and the ICP’s PPPs are different, not only in time-space dimensions, but also in scope. A national CPI
10

www.imf.org/en/Data/Statistics/cpi-manual

11International Labour Office (ILO), International Monetary Fund (IMF), Organisation for Economic Cooperation and Development (OECD), Statistical Office of the European Union (Eurostat), United Nations
Economic Commission for Europe (UNECE), and the World Bank.

6

covers consumer goods and services purchased by households, while the ICP covers the wholeexpenditure side of GDP, and its analytical components, including household consumption.
Furthermore, subnational PPPs can be constructed by comparing prices across regions and locations
within a country in the same period.
National CPIs are some of the most important economic indicators and are compiled anddisseminated, on a regular basis by NSOs or other producers of official national statistics around theworld. CPIs have a long history and Chance (1966) reviews the emergence of the concept of priceindices from the 1600s onwards including William Fleetwood’s measure of price change in 1707. The
Laspeyres index developed in 1871, and its derivatives, is still one of the most widely used index
number formula which, in concept, measures the change in the cost of a fixed basket of commodities
in the reference period had that same basket been purchased in any succeeding or preceding
periods. The CPI Manual discusses the types of index used in the compilation of CPIs including theLowe index, the Young index, and modified versions of these. In addition to being used as a
macroeconomic measure of inflation for a given country, CPIs serve as key inputs into monetary
policy decisions and play a prominent role in monitoring the effects of government policies. In
addition, the CPI is used to index monetary payments, including salaries and wages, to ensurecompensation adequately reflects changes in the cost of living, government payments such as social
security and rent contracts, and other payments. The CPI also provides the general public with ameasure of changes in the prices of goods and services consumed. CPI compilation methods
continue to evolve over time to reflect improvements and innovations to data sources and collection
methods, index calculation formulas, and user needs. Current recommendations and best practices
are outlined and discussed in the CPI Manual.
PPPs are both currency conversion factors and spatial price indexes. They are calculated based on
prices for a common basket of goods and services in each participating economy. They convert
different currencies to a common currency and, in the process of conversion, equalize theirpurchasing power by eliminating the differences in price levels between economies. They show, withreference to a base economy, the relative price of the given basket of goods and services in each of
the economies being compared.
PPPs, as measures of spatial price level differences, are a relatively new development that can betraced back to Gilbert and Kravis (1954) and the pioneering efforts of Kravis in establishing theInternational Comparison Project (ICP) at the University of Pennsylvania in 1968, as a research
program in collaboration with the United Nations Statistical Division (UNSD). The ICP has grown froma small research project covering ten countries in 1970 into a global statistical initiative with 176
economies fully participating in the ICP 2017 cycle, and was instituted by the UNSC as a permanentelement of the global statistical work program at its 47th session in 2016. The ICP website provides
a wealth of information on the concept, calculation and use of PPPs, as well as the history andgovernance of the program. Furthermore, the World Bank (2013) publication “Measuring the RealSize of the World Economy: The Framework, Methodology, and Results of the International

7

Comparison Program (ICP)”,12 provides a comprehensive overview of the compilation of PPPs, and
the World Bank (2015) publication “Operational Guidelines and Procedures for Measuring the RealSize of the World Economy 2011 International Comparison Program”13 provides detailed guidelines
on the operational aspects of ICP.
Despite conceptual similarities and closely related data collection requirements, historically there
has been very little harmonization of the national CPI and PPP compilation activities of national
statistical offices with the two measures typically constructed under different production
infrastructures. However, this has started to change recently, and this guide aims to identify thepractical steps that can be taken to integrate collection of consumer prices for both activities.
Achieving this is vital to the sustainability of the ICP as a permanent UN exercise and key to improving
the accuracy of national CPIs. In this way the ICP and CPI data collection processes can be seen as
symbiotic – working together to improve and sustain each other.
The challenge of CPI-ICP integration is to incorporate elements of the ICP methodology that will
enhance the quality of the national CPI while not compromising the objectives, representativity, andreliability of the CPI. In addition, meeting this challenge, and successfully integrating parts of the twoprocesses, will create an ideal data set for the computation of subnational PPPs.

4. Benefits of CPI-ICP integration
Similarities in the conceptual and operational aspects that underpin CPI and PPP compilation are
indicative of significant synergies and benefits from an integrated approach to these two activities.
It is important to recognize the dual nature of the integration process which can simultaneously
benefit both programs, resulting in improvements in CPI compilation and at the same time
contributing to the sustainability of international price comparisons.

Benefits for the CPI
There are significant benefits to NSOs adopting an integrated approach to the compilation of CPI and
PPPs for ICP. In the case of developing countries with limited statistical infrastructure, integrating
the activities of the ICP with those of the CPI can provide an opportunity for NSOs to maximize
efficiencies and make the case for additional resources to support both programs.
•

Participation in the ICP can help strengthen statistical infrastructure and enhance institutional
capacity. Participation in the ICP is likely to identify deficiencies which can help improve
management and planning functions such as the recruitment and training of new staff for
conducting general price surveys as well as household expenditure surveys. Furthermore, thevalidation procedures used by the ICP can be utilized to improve the quality of national statistics.

12

https://www.worldbank.org/en/programs/icp/brief/2011-icp-book13

https://www.worldbank.org/en/programs/icp/brief/2011-operational-guidelines8•Participation in the ICP will require a review of data collection methods and index processing
systems, and this review could identify areas for improvement. Designing and conducting sample
surveys to international standards can be a demanding exercise. Participating countries mayneed to develop systematic plans to augment CPI survey designs to integrate the ICP to meetthe needs of both programs, without compromising the quality of either.

•Computerized data-processing systems may need adaptation and enhancement to incorporate
ICP specifications. The adoption of detailed ICP specifications into CPI necessitates acomputerized data processing system in order to accommodate more information. The process
may speed up the adaptation of similar computerized data processing systems to CPI.

•

As the ICP is a highly structured and harmonized activity, participation in the ICP could
potentially encourage the implementation of harmonized CPI methods at the regional level.

•

The framework for the ICP may be adapted for making price level and PPP-based income andexpenditure comparisons across areas and regions within a country. There is growing demand
for subnational PPPs which can help assess the levels and movements of real incomes andconsumption in different regions. Intra-country price comparisons are helpful in monitoring
progress towards the United Nations Sustainable Development Goals (SDGs) and monitoring theincidence of poverty in different parts of the country.

Benefits for the ICP
Conducting the ICP can be challenging for participating countries because NSOs traditionally facebudgetary and human resource constraints. ICP-related activities are typically considered secondary
and are over and above the regular activity of compiling the national CPI and national accounts.
Meeting the requirements of the ICP and adhering to schedules for the collection and provision ofreliable price data may stretch the limits of those departments involved in these activities. The
dependence of many countries on additional funding needed to participate in the ICP, specificallyregarding the collection of price data, is a major challenge to the sustainability of the ICP. It is in thiscontext that the benefits of integrating ICP and CPI activities should be considered.
•

The absorption of ICP activities into the mainstream CPI activities of NSOs will help to ensure theallocation of national resources for the ICP and may provide an opportunity for NSOs to request
additional resources for both exercises. This, in turn, will help to sustain the ICP globally andresult in a more enthusiastic participation by countries in the ICP.

•As the ICP strives to increase the frequency of its benchmark comparisons or cycles by adopting
a rolling price survey approach, the program will depend on reliable estimates of domestic
inflation for major household consumption categories.14

14

Both the CPI and the ICP use the Classification of Individual Consumption by Purpose (COICOP).

9•

Domestic rates of inflation based on the CPI and for broad categories of household consumption
can be used to validate price data collected for the ICP in different cycles. The implied inflation
rates for common items in two consecutive ICP cycles can be assessed against domestic inflationin the closest related component of CPI. The deviations in the directions and magnitudes
between the inflation rates in ICP and in CPI can be informative in validating price data collected
for the ICP.

•The quality of ICP data is likely to improve if price surveys for the ICP are integrated with CPI
price surveys. The expertise of CPI price collectors, especially on local markets, is likely to
improve the quality and reliability of prices collected for the ICP.

•The CPI basket of goods and services may be used to identify ICP products that are“representative” or “important” for a given country. The current ICP methodology accords
higher weights to the prices of products that are deemed to be important.

•

The integration of ICP price surveys with those of the CPI would maximize efficiencies and may
result in budgetary savings in staff, costs, and resources.

•

The ICP relies on national accounts expenditures for the benchmark year as weights in thecomputation of PPPs. Expenditure weights for household consumption aggregates are usuallybased on CPI weights which are compiled using data from household expenditure surveys.

The potential benefits stemming from the increased integration of CPI and ICP activities discussed
above are at the core of the recommendations made in this guide.

5. Current level of integration of CPI-ICP activities
As part of the preparations for this guide on CPI-ICP integration, it was necessary to first gauge andunderstand the level of integration currently achieved in countries that are actively participating in
the ICP. After deliberations of the Task Force assigned with the drafting of this guide, and thesubsequent endorsement by the ICP Technical Advisory Group (TAG), it was decided to conduct asurvey, using a short questionnaire (in Appendix A), of the current state of CPI-ICP integration in2018 and 2019. The survey was administered through the national implementing agencies of theparticipating economies in the ICP 2017 cycle.15 The questionnaires were sent via email and the
response rate was 67 percent with responses received from 111 countries. The distribution of
responses by regions is shown below in Table 5.1.

15

Results of this survey are show as per the ICP administrative regions.

10

Table 5.1: Distribution of responses to questionnaire, by ICP region.
No. ofResponse rate
responses
Total
111
67%
Africa22
44%
Asia and the Pacific
22
100%
CIS
8
100%
Eurostat24
65%
Latin America and the Caribbean
24
67%
Western Asia
1192%
An important factor in the integration of CPI and ICP activities is the frequency of price collection.16
If the frequency is similar, integration may be easier to achieve. The survey finds significant
differences in the frequency of price collection for the CPI and ICP. As expected, CPI price collection
is mainly carried out on a monthly basis, whereas ICP price collection takes place either monthly orquarterly (Table 5.2).

Table 5.2: Frequency of price collection.
Total
Monthly
Quarterly
No response

CPI
111
106 (95%)
2 (2%)3

ICP
111
41 (37%)51 (46%)19

Around 65 percent of responses indicated that different forms are used for CPI and ICP pricecollection (Table 5.3). Product specifications used in the ICP forms are detailed and based on
Structured Product Descriptions (SPDs),17 which are not generally used for CPI price collection.

Table 5.3: Is the same form used for CPI and ICP price collection?
Total
111
No
71 (64%)Yes
31 (28%)
No response
9 (8%)

16

For ICP, this refers to the frequency of price collection during a reference or benchmark year, except for theEurostat-OECD PPP Programme which relies on the rolling survey approach.
17

Structured Product Descriptions (SPDs) are generic descriptions of items included in the ICP basket for pricesurveys. The SPDs are essentially a list of price-determining characteristics relevant to a particular narrow
cluster of products.

11

One of the main reasons for not using the same form for both exercises is due to differences in the
product lists used for the CPI and the ICP. The responses summarized in Table 5.4 show that the
average reported overlap between the CPI and ICP baskets is as low as 33 percent with a high 25.7
standard deviation. Four responding countries reported no overlap between the ICP and CPI baskets.
These responses need to be cautiously interpreted due to possible measurement errors arising outof misinterpretation of the question. As the ICP lists at the regional level are constructed with active
input from participating countries, it is very unlikely that there is zero overlap between the ICP and
CPI baskets. It is possible that a response of no overlap could mean that no prices collected for CPI
were used for ICP, or the ICP item specifications are so narrow that they do not match exactly withthose of similar items in the CPI basket.

Table 5.4: Percentage of ICP products included in the CPI basket.
Observations
Minimum
Maximum
MeanMedianStandard deviation

94
0%
85%
33%
32%
26

With regards the timing of price collection activities (Table 5.5) 30 percent of respondents reported
that the CPI and ICP surveys are conducted at the same time, while another 32 percent indicatedthat the timing of surveys is synchronized for collecting prices for items that overlap between thetwo baskets. A possible explanation for the 36 percent of the responding countries that reported
separate times for ICP and CPI price collection may be due to differences in the frequency of pricecollection. Responses on the frequency of price collection showed that 95 percent of respondents
collected CPI prices monthly whereas only 37 percent collected ICP prices on a monthly basis (Table
5.2).
Table 5.5: Is price collection for the CPI and ICP carried out at the same time?
Total
111
Yes
33 (30%)No
40 (36%)
Only overlapping items
36 (32%)
No response
2
Turning to data validation, editing, and processing activities, there are encouraging signs of CPI-ICP
integration. Table 5.6 shows that around 40 percent of the responding countries use the same dataprocessing system to capture data (“yes” + “only overlapping items”), but 60 percent of countries
currently do not.
12

Table 5.6: Do the CPI and ICP use the same computer system to capture data?
Total
Yes
No
Only overlap
No response111
24 (22%)62 (56%)23 (20%)2

Table 5.7 shows that 40 percent of countries indicated that they use the same editing proceduresfor their CPI and ICP data, and an additional 20 percent indicated that they use the same procedures
for analyzing price data for overlapping items. As editing procedures used for ICP are specificallydevised for validating cross-country data, it is interesting that these methods are being applied in
the CPI of 60 percent of countries.

Table 5.7: Do the CPI and ICP use the same editing procedures?
Total
111
Yes
46 (41%)No41 (37%)
Only Overlap
22 (20%)No response2
Sixty-nine of the responding countries (64 percent) indicated that ICP validation procedures havehelped in processing CPI price data, some significantly and others to a lesser degree (Table 5.8). The
countries where ICP validation procedures have played an important role are mainly from Africa,
Asia and the Pacific, and to a lesser extent from Latin America and the Caribbean and Western Asia.
These responses suggest that participation in the ICP has contributed to statistical capacity building
in many countries with respect to the editing and validation of CPI price data.

Table 5.8: Do ICP validation procedures help with the processing of CPI data?
Yes, significantly

Yes, somewhat

No

25

44

39

Africa985Asia and the Pacific

9

11

2

CIS

0

3

5

Eurostat0616

Latin America and the Caribbean

2

12

9

Western Asia

5

4

2

Total

13

More than half of responding countries indicated that ICP staff are drawn from the staff dedicatedto CPI compilation (Table 5.9). An additional 27 percent of the countries indicated an overlap
between ICP staff and CPI staff.
Table 5.9: Are ICP staff a subset of CPI staff?
Total
111
Yes
59 (53%)No
20 (18%)
Some are and some not
30 (27%)
No response
2
Finally, 65 percent of the countries have plans to improve CPI-ICP integration in the future (Table
5.10). Outside of the Eurostat-OECD PPP Programme, 64 out of 84 countries in the remaining regions
indicated plans for further integration of the CPI and ICP exercises.

Table 5.10: Does your organization have plans to improve CPI-ICP integration?
Yes

No

Total
AfricaAsia and the Pacific
CIS
Eurostat
Latin America and the CaribbeanWestern Asia

37
6
4
5
17
5
0

69
15
18
3
5
18
10

In summary, responses from ICP participating countries show some signs of CPI-ICP integration andindicate plans for further integration of CPI-ICP activities. The contents of this guide are expected tohelp these and other countries to achieve higher levels of CPI-ICP integration, and thereby benefitingfrom synergies and enhanced efficiency in the compilation of CPI and PPPs.

6. The Statistical Value Chain as a basis for CPI-ICP integration
A systematic approach to identifying areas of possible alignment and integration is to use thestatistical value chain (SVC), which is also known as the statistics lifecycle, statistical process model,
or data value chain.18 The international statistics community has formalized the SVC in the Generic18

See https://opendatawatch.com/publications/the-data-value-chain-moving-from-production-to-impact/
for a detailed explanation on statistical value chains.

14

Statistical Business Process Model (GSBPM).19 The SVC details the generic sequence of steps required
when undertaking a statistical activity and is comprehensive in scope, covering all possible elementsof a process.
The main headings applicable to the practical aspects of integrating ICP and CPI activities arehighlighted in Figure1 and are expanded on below.
Design refers to the initial conceptualization and setup of the survey. For repetitive surveys such as
price collection, design takes place on initiation of the activity and again only when improvements
are planned. A large proportion of this guide is dedicated to the design phase as it is here that muchof the effort is required.
Collection refers to the physical activities involved in recording data including the planning of pricecollection, the training of collection staff, and first line quality assurance processes. The standard
SVC sample selection step forms part of the collection phase; however, in this guide it is covered
under Design as product identification and the respective specifications are central to the overall
conceptualization of integrated price collection.
The Processing and Editing phase refers to activities such as the editing and validation required toclean the data so that they are fit for analysis.
Figure 1: Statistical value chain

6.1 Design
6.1.1 Classification
Key recommendation: COICOP classification and hierarchy should be consistent for ICP and CPI
products.
The Classification of Individual Consumption According to Purpose (COICOP), which was updated in2018, forms the basis for organizing product data in both the CPI and ICP. Compliance to theinternational high level COICOP structure by countries facilitates CPI-ICP integration as well ascomparisons of national CPIs.

19

Information on the GSBPM is available at https://statswiki.unece.org/display/GSBPM/GSBPM+v5.1

15

The official COICOP manual20 defines the product hierarchy down to the level of a subclass, forexample, “bread and bakery products”. Many countries create further levels themselves, the first ofwhich is the micro-class which is equivalent to the ICP’s basic-heading level. Typically, expenditure
data at this level can be used to develop more detailed elementary aggregates in the CPI (for example“bread”) than the subclass identified by COICOP. It is important for ICP purposes that the lowest, or
most elementary aggregate, level used in the CPI can be mapped to the ICP basic headings. It should
be noted, however, that the elementary aggregate level differs from country to country based on
the availability of data, and many countries lack the data needed to develop weights at such a
detailed level.
Table 1 shows the levels of the classification hierarchy in the new COICOP version, a typical CPI
breakdown, and the ICP hierarchy. This table helps readers to understand the terminology employedby both the CPI and ICP as well as in this guide. The formal COICOP classification only goes to thelevel of class (COICOP 1999) or micro-class (COICOP 2018). It is important to note that the term
“item” is used to describe the lowest level of classification at which the CPI typically, though notalways, has expenditure weights. This is the elementary index level which corresponds to the basic
heading level. The term “variety” is used to refer to the actual product in a specific store that is being
priced.

Table 1: COICOP, CPI and ICP hierarchy21
Hierarchy
CPI

ICP

COICOP 2018

DIVISION

DIVISION

CATEGORY

Group
Class

Group
Class

Group
Class

Subclass

Subclass…

Micro-class
Item

Basic
heading
Item

Variety…

COICOP
2018

ExamplesCOICOP 1999

ICP

FOOD AND NONALCOHOLIC
BEVERAGES
Food
Cereals and cereal
products

FOOD AND NONALCOHOLIC
BEVERAGES
Food
Bread and cereals

FOOD AND NONALCOHOLIC
BEVERAGES
Food
Bread and cereals

Bread and bakery
products
Bread

…

…

Bread

Bread

Loaf of white bread

Loaf of white bread

X brand in y outlet

X brand in y outlet

Bread loaf, white,
Brand not relevant…

20

https://unstats.un.org/unsd/classifications/business-trade/desc/COICOP_english/COICOP_2018_-_preedited_white_cover_version_-_2018-12-26.pdf
21

The CPI example in this table assumes the country in question has an elementary aggregate or “item” of“Loaf of white bread”. However, the level of elementary aggregates can vary between countries.

16

COICOP
2018

CPI

ICP

DIVISION

DIVISION

CATEGORY

Group
Class

Group
Class

Group
Class

Subclass

Subclass…

MicroclassBasic
heading

Item

Variety

COICOP 2018

COICOP 1999

ICP

CLOTHING AND
FOOTWEAR
Clothing
Garments

CLOTHING AND
FOOTWEAR
Clothing
Garments

CLOTHING AND
FOOTWEAR
Clothing
Garments

Garments for women…

[Women’s skirts anddresses]

Garments

Item

Garments for
Women or girls
Women’s shirts,
blouses, pullovers,
sweaters, etc
Women’s shirt

Women’s skirt…

X brand in y outlet

X brand in y outlet

Women's pullover,well-known brand,low…

6.1.2 Specifications
Key recommendation: The concept of Structured Product Descriptions (SPDs) can guide CPI
compilers to ensure the development of detailed and complete variety specifications that support
both CPI and ICP needs.
The list of product specifications to be recorded is crucial for price data collection and forms thebasis for most of the recommendations in this guide. Product specifications are the item
characteristics that influence the price of the item (price-determining characteristics). These appear
as a set of fields in a questionnaire or database in which information is recorded, each field having
either a predetermined range of responses (for example, Yes or No) or rules that govern theinformation to be recorded.
The product specifications describe the observable price-determining characteristics that need to bematched when a variety of an item is selected to be priced. The benefits of this approach include thefollowing:•••a detailed description of each collected variety enables confirmation that the priced variety
meets the parameters of the elementary index level product;a detailed description provides adequate information for the price collector to find precisely
the same variety at the subsequent collection period, usually a month later; anda detailed description may also include multiple fields covering different aspects of the
product which allows a greater range of data for comparison and analysis purposes.

The CPI Manual refers to tight and loose specifications of product or item varieties. Tight
specifications better facilitate the calculation of average prices and ensure comparability of items
across countries, which is the aim of ICP price collection. Accordingly, the ICP stipulates strict
requirements for the selection of items for pricing. These may include restricting the choice of brand17

or the size of an item by weight, volume, or quantity for example. For the CPI, tight specifications
tend to be useful for electronic and other items with high rates of turnover.
Loose specifications ensure that the CPI variety sample broadly reflects differences in regional tastes
and preferences, as well as differences due to socioeconomic factors. As noted in the CPI Manual,
the individual varieties sold differ from outlet to outlet, and from region to region, and the CPI shouldreflect these differences. In general, loose specifications can be useful for food, beverages, clothing,and personal items.
With loose specifications, the price collector applies a set of rules to identify a representative variety
within the sampled outlet the first time the variety is selected for pricing. Typically, these rules are
aimed at identifying the ‘volume seller’ within certain parameters, that is the most popular variety.
Adequate descriptive information must be recorded to ensure the identical variety is priced insubsequent periods.
National CPIs vary in the extent to which item selection is prescribed through loose or tightspecifications. In some countries, all descriptive characteristics are captured in only one or two fields,
which may facilitate the future identification of the variety for price collection. However, such a loose
specification may capture a widely varying set of items and prices, reducing the data’s usefulness foranalysis and the calculation of average prices. It is critical that the detailed specifications include all
price-determining characteristics.
The CPI can potentially benefit from the characteristics used to define items in the ICP to ensuresufficiently detailed variety specifications are used in the price collection forms and captured in therecording database. However, it should be noted that while tight specifications are appropriate to
the ICP as the price collector is directed to a specific variety, looser specifications for the CPI canprovide a more broadly representative index. In cases where an ICP item is not already included in
the CPI list, the CPI variety sample can be augmented to include ICP items. Care must be taken,
however, to ensure that these additional varieties are representative from the perspective of the
CPI, or they should be excluded from CPI calculation and sample design.
When using loose specifications, compilers should ensure that specifications include all pricedetermining characteristics and provide additional fields to include other descriptors, as needed.
This practice can improve the quality of the CPI by assuring the correct matching of item varieties
and enable deeper analysis of price trends for quality assurance purposes. Providing more extensive
descriptive variables will make it easier to observe whether a variety priced in the CPI is similar to an
item in the ICP list.
The specifications required to define the characteristics will differ according to the type of product.
The SPD approach used by the ICP provides an excellent standard for the coverage of specifications
required for correctly describing an item to be priced. It also provides the range of options requiredby the ICP for precise identification and description, and greatly assists when changes or updates toitems are required. Importantly, the specifications should map clearly to the fields used in the editing
and imputation processes.
18

Standardized characteristics will vary by ICP basic heading. Using these standard characteristics,
countries can include additional details on CPI data collection questionnaires. These can bepresented either as tick boxes to mark whether a particular characteristic is present or not, or as
open fields to record information about the characteristic. This will allow greater identification of
the overlap between the ICP and CPI item lists.
Where possible, a predetermined set of responses should be used to ensure consistency in recording
characteristics. Where mobile devices are used in price collection, drop down lists could be created.
Having more characteristics recorded is preferable, but this will depend on the resources andcapacity of the NSO.
Priority should be given to those price-determining characteristics that are essential for like-withlike price comparisons and which feature in the ICP validation process. Priority characteristics are:
•
•
•

Brand
Size/weight/quantity
Unit of size

Figure 2 illustrates how specifications required for the ICP could be mapped to a smaller number of
specifications in the CPI collection form, and how they could be used for reporting for ICP purposes.
The column labelled “ICP restrictions” represents the parameters determined by the ICP to ensure
consistency across countries. ICP specifications are the details of the specific item to be priced asrecorded on the SPD.
Figure 2: Mapping of ICP restrictions, variables, and CPI variables for BreadType of brand(SB/WKB/BL/BNR)

Note: The “Brand” box in the second column may convey the impression that the ICP typicallystipulates a specific brand (SB) to be priced. However, such cases are more of an exception than the19rule. In most cases the ICP asks for items that are either a “well-known brand” (WKB), “brandless”(BL) or “brand not relevant” (BNR); this gives the country the flexibility to decide which brands toprice, albeit within all other SPD parameters.

6.1.3 Price collection forms
Key recommendation: CPI and ICP data should be recorded on the same price collection forms,
which must contain all the specifications identified as necessary for both CPI and PPP compilation.
The contents of price collection forms should include the price-determining characteristics that havebeen identified. Neither the ICP nor the CPI Manual are prescriptive on the layout of price collection
forms and whether they should be paper or electronic. Good practice principles in questionnaire
design include: (i) ease of use; (ii) clarity in identifying the correct variety; (iii) adequate space torecord information; and (iv) ability to extract information. The CPI Manual provides an example of aprice collection form which will enable CPI-ICP integration.
There should be no difference in the CPI and ICP collection forms and, ideally, there should only be
one set of forms on which price data are recorded. In many cases, CPI collection forms are
spreadsheets with all the information for each variety contained in a single row; however, this is not
ideal for either the CPI or ICP as the forms may not contain all the fields needed for ICP collection. A
review and redesign of forms will be required to improve CPI questionnaires and to describe theproducts to the standard required for the ICP. The layout of the form should incorporate theadditional variables in a logical flow for recording, rather than being inserted as an ‘add on’.
Increasingly countries are using mobile devices such as tablets to collect CPI data. These have theadvantage of combining data collection and quality control. Data collection via mobile devices
provides an opportunity to create standardized responses to each variable in the form of drop-down
lists. This avoids differences in responses between data collectors and enables the inclusion of simple
standards such as the word order of description and spelling. Such standardization will significantly
reduce editing and validation efforts as well as improve the extraction of data.
Figure 3 provides an example of how the characteristics required for both the CPI and ICP can easily
be incorporated into a single data collection form. As discussed earlier, the specific characteristics
will differ according to the particular micro-class (CPI) or basic heading (ICP).

20

Figure 3: Example price collection form with necessary specifications for both CPI and ICP
purposes
Prices forJune 2018
Collection Date6 June 2018
Price collector XXX

Location Megacity

Class

COICOP

Bread and Cereals

OutletItem information
Price number Basic heading Item
Item code
Brand
01
Bread
Brown
01113.1 Albany
bread
02
Other cerealsCornflakes
01114.1 Kellogs

Super store

Type of outlet

Supermarket

0111
Product description/presentation
Product name Product code
Ingredients
Packaging
Premium
01113.1.01 100 % wheat flour Plastic bag
brown bread
Kellogs corn flakes
01114.1.01 Maize/ corn
Cardboard box

Further description Size
Red and blue
700
colouring.
Picture of bowl 500
with cereal androoster.

Price information
Unit of size Previous price Current Price Comments
gram
gram

03
04
05
06
07

6.1.4 Sample design and selection
Price surveys require samples to be selected at four different levels. Each of these provide
opportunities to integrate the ICP needs into the CPI data collection process:••••first, there is the item level which defines the set of representative products that make up
the index;
second is the geographic sample which determines the areas where prices will be collected;
third is the outlet sample which identifies the specific stores or outlets where prices will becollected; andfourth, there is the sample of varieties, which form the actual survey data.

a) Product (item) list/basket
Key recommendation: The CPI basket should be used as the basis for the ICP product sample withthe goal of minimizing the additional products needed for the ICP.
There are three product lists which impact the selection of products to be priced by each country for
the ICP. They are:•••the national CPI basket;
the global ICP product list;22 andthe regional ICP product list.

22

The global ICP product list is same as the list of global core items, which are priced for the purpose ofproviding a link or overlap between regional comparisons at the basic-heading level, in order to combine theregional comparisons into a single global comparison. (World Bank, 2020, page 199).

21

The aim of the regional list is to have as many products as possible priced in the countries affiliatedwith that region. This provides the basis for spatial price comparisons.
Each regional list incorporates a number of products that are included in the global ICP product list,
as well as purely regional products. In the ICP 2017 cycle each regional list incorporated around aquarter of products from the global list, and the choice of products differed between regions.
Typically, a third of a country’s national CPI basket is made up of products that are on the regional
ICP product list. To fulfill ICP requirements, the country selects additional products to price from therelevant regional ICP list. Together these additional products and those already collected by the CPI
make up the national ICP list (Figure 4).
Figure 4: Relationship of national CPI and regional and global ICP product lists

Overlap products

In the case of the CPI basket, the main criteria for inclusion of products or items is representativity,
primarily based on the share of household expenditure typically spent on that product or item.
Relative volumes sold may also be considered in establishing the representativeness of a product.
This list will remain fixed until new expenditure data are available or if selected products disappear.
The list of products for the ICP, however, is first established by the regional implementing agency incoordination with national implementing agencies and in conjunction with the ICP Global Office.
Countries are expected to price a subset of the ICP list based on their availability as well as theirrepresentativity, which is indicated through importance indicators.
The national implementing agency should actively participate in the preparation of the ICP productlist for the region and include the national CPI product list in the deliberation process. This is only
possible when information on the specifications or price-determining characteristics of CPI products
is detailed and available for consideration.

22

This process should be hierarchical, as follows:•First, the national CPI basket and product list along with SPDs are determined based on countryspecific needs and considerations. It is critical to retain the integrity and relevance of the CPI
basket while striving to achieve integration of CPI and ICP activities.

•

Second, lists of CPI products from different countries are used as inputs into the preparation ofregional ICP product lists.

•

Finally, regional ICP product lists are used as inputs into the final global ICP product list.

Countries define their baskets at different levels of classification and may use terminology relevantto their context. Thus, some cross referencing to a common standard will be required. The ‘item’
level is most appropriate (see Table 1). The following is an excerpt from the ICP 2021 global list.

Basic Heading: Bread
Item Code

Item Name

110111301

Baguette, BNR

110111302

Bread loaf, wheat, BN

110111303

Bread loaf, whole wheat, BNR

110111304

Bread loaf, rye, BNR

110111305

Bread roll, BNR

110111306

Bread loaf, sliced, WKB

Note: BNR=brand not relevant. WKB=well-known brand (in the country)

Practical application:
A simplified process for selecting the regional basket could be:•

Countries submit a list of items included in the CPI basket along with their detailed productcharacteristics to respective ICP regional implementing agencies;•

ICP regional implementing agencies match products to determine the scope of commonality;•

A cut-off point is identified whereby the regional list comprises products with a certain
percentage of country commonality;•

The regional list of items is then compared against previous regional lists and the global list
to ensure adequate continuity and coverage; and•Countries decide which of the regional list items they will include for the additional ICP
collection based on considerations of importance and representativity within each ICP basic
heading.

23

Example
Global items

Country 1

Country 2

Country 3

Country 4

Baguette, BNR
xBread loaf, wheat, BNRxxxx
Bread loaf, whole wheat, BNRxxx
Bread loaf, rye, BNR
xBread roll, BNRxxx
Bread loaf, sliced, WKB
x
Note: BNR=brand not relevant. WKB=well-known brand (in the country)

Country 5

RatingSelected
Y/N

x
x
x
x

1
5
4
2
4
1

N
Y
Y
N
Y
N

In this example, five countries identified those varieties included in their national CPI from a list of
six products which comprise the global list under the basic heading “Bread”. Then, the region
decided to include all products that appear in the national CPI basket of at least three countries. This
resulted in three products (“Bread loaf, wheat, BNR”; “Bread loaf, whole wheat, BNR”; and “Breadroll, BNR”) being chosen to form part of the regional basket. Only one of these – “Bread loaf, wheat,
BNR” – appears in the national baskets of all countries. Those countries that do not have these three
products in their national baskets may consider them when selecting their additional ICP collection.
Similarly, there may be products that are significant for many countries in a region but do not form
part of the global list. This process will help to identify those items that should form part of theregional basket.
It is possible that this process is implemented in two stages. The first takes place at a sub-regional
level (in Africa this would be Southern African Development Community (SADC), Common Market
for Eastern and Southern Africa (COMESA), Economic Community of West African States (ECOWAS),etc.) and the sub-regional lists are consolidated into a regional list using a similar approach.
The aim is to obtain a balance between ensuring adequate numbers of products to enable the ICP
comparison and ensuring that the NSOs can operationally cope with the additional collection.
Once the regional product list has been determined, countries will face three possible scenarios:
1. Products appear in the regional ICP list which are not in the national CPI basket. While it may bepossible for some countries to introduce ICP products into the collection process at any time,
resource permitting, it should be feasible for additional ICP products to be incorporated into CPI
data collection activities, for instance, when the CPI basket is being updated. These additional
items can be excluded from the actual CPI compilation phase.
2. There is overlap between the ICP and CPI product lists and the detailed specifications are similar.
This is the ideal situation as there is seamless integration.
3. There is overlap between the ICP and CPI product lists, but the items are not easily mapped. For
example, where a national CPI has a category “women’s shirts” at the elementary index level.
Here it will be necessary to ensure that there are an adequate number of varieties in the sample
that comply with the ICP specifications. So long as the information for all the characteristics as24

discussed above are collected, compliant varieties can be identified and extracted from the data
after collection.

b) Geographic sample
Key recommendation: CPI and ICP prices should be collected in the same geographic areas, to theextent possible.
Conceptually both the CPI and the ICP should have national coverage. Both aim to be representativeof the goods and services purchased by households within the country. In practice, the CPI
geographic sample in many countries comprises only urban areas. There are several reasons for this.
First, urban areas typically have a higher share of household expenditure and total transactions.
Second, residents of rural areas often travel to the nearest urban area to purchase semi-durable anddurable goods (for example, clothing, appliances, mobile telephones, etc.) as well as services. Third,
if the price changes in rural and urban areas are the same, the price changes in the urban areas are
considered to be representative of the entire country. Finally, when relatively few goods and services
are available for pricing in rural areas and outlets are few in number and located great distances
apart, the collection of prices in rural areas is not efficient from a resource perspective. Because of
the additional financial and logistical costs of data collection in rural areas, in some countries the ICP
data collection covers more areas than the CPI, while the converse is true in other countries.
Resources available to NSOs for the collection of prices, and whether relative price changes are thesame in rural and urban areas, drive decisions regarding the collection of prices in rural areas. Whenpossible, the areas of price collection should be as geographically representative of the wholecountry as possible.
In larger countries, it may be possible to use spatial adjustment factors (SAFs), similar to those usedin the Eurostat region, to convert capital city prices into national average prices. SAFs require thecompilation of intra-country spatial price index numbers at frequent intervals.

c) Outlet sample
Key recommendation: The CPI outlet sample should include all outlet types where ICP products areavailable.
Ideally the CPI outlet sample should include all different types of outlets and should broadly
represent those outlets where households make purchases. Similarly, the average prices requiredfor PPPs should be based on prices collected from outlets representative of where such items arepurchased. Here there is a natural synergy between the CPI and ICP. In those cases where an item in
the additional ICP collection is not available in any of the CPI sampled outlets, it may be necessary
to augment the CPI sample to include those outlets where the ICP item(s) are available for pricecollection.

25

Where a listing of outlets, such as a business register or administrative records, exists then outlets
can be sampled according to standard statistical methods. However, in the absence of a formal
sampling frame, the choice of outlets is often left to the judgement of CPI staff or the price collector
who has some knowledge of the outlets where each product type is predominantly sold orpurchased.

d) Sample of product varieties for pricing
Key recommendation: CPI sampling methods should include ICP requirements.
The selection of actual product varieties for pricing is perhaps the most important aspect ofcollecting price data as these observations form the records from which all calculations are madepossible.
Typically, sample selection uses non–probability or purposive methods. For the CPI, the goal is toselect a variety sample that broadly represents expenditures by households in a given country. Oneof the most common methods used is to select the most popular varieties. The price collector follows
certain rules and procedures to ensure representative varieties are selected and defined. For tightspecifications, the variety is selected and defined by CPI staff, sometimes in consultation with datacollectors. The detailed specifications are then sent to all data collectors for the collection of prices.
ICP sampling requirements are designed to compute average prices and therefore comparability isrequired before representativity. To this end, clear requirements on the type of brand and size apply.
This will cause some difficulty in aligning CPI and ICP price collection. Representativity informationin the ICP is conveyed through importance indicators. Countries report whether an item is important
relative to the other items within a particular basic heading (not across all basic headings). Countries
should take this process seriously and provide this information for ICP purposes. Importance may be
determined by the presence of the product (or one similar) in the CPI basket (overlap) or by what
price collectors perceive as the best-selling products.
For those ICP products that do not form part of the CPI basket and are included in the additional
price collection, the number and specifications of the priced products will follow the requirements
set down by the ICP as far as possible.
For overlap products, there are two possible scenarios:
1. ICP product specifications are compatible with the most representative products included in
the CPI sample.
2. ICP needs require the identification of a variety not considered representative for CPI needs.
Figure 5 illustrates these scenarios and decisions in a flow diagram.
In the first case, all that is required is that those CPI products that conform to ICP requirements are
identifiable and that all the relevant characteristics are recorded. Section 6.1.2 discusses thevariables that should be included on the questionnaire and data processing system to ensure that26

these products can be easily extracted and incorporated into the calculation of average prices for
PPPs. Identification of the variety could be through a parallel coding system, or a simple tick box.
In the second case, the price collector should be provided with a list that details the specifications
for each product. Such lists could form part of the product catalogue discussed later in section 6.2.2.
The prices of these products could be used in calculating the CPI as long as other, more
representative varieties are priced and the number of less representative varieties included accounts
for a small proportion of the total number of collected prices for that item.

Figure 5. Scenarios for sampling ICP product varieties

As part of the ongoing sample maintenance processes for CPIs, NSO head office coordinators willneed to review the number of prices collected for the ICP each month to ensure that the sampling
requirements are met. Under- and over-collection should be recorded in order to provide remedial
instructions to collection staff in the following month.

6.2 Price collection
Price collection is where integration is effectively achieved at an operational level. This is especiallythe case where ICP price collection has been funded and operated separately from the CPI. As the
ICP becomes a more regular annual activity, additional external funding is likely to become scarcer.
There is significant diversity in the way that countries arrange their CPI price collection operations
including the relationship between statisticians and the field workers. In most cases, ICP datacollection will form part of the CPI operations and there will be no need to make major changes.
27

Integrating the ICP should prompt a broad review of CPI data collection methods to identify areas
for improvement.
There are two main areas in which ICP and CPI integration can be promoted in the data collection
phase. The first is in the daily operational processes managed through a schedule and work
allocation. The second is in the training and support provided to data collectors.

6.2.1 Schedule and work allocation
Key recommendation: All price data should be collected according to the same schedule.
•

All price collectors should be involved in collecting all prices irrespective of whether they areused for the CPI only, both the CPI and the ICP, or for the ICP only.

•

To maximize efficiencies, the collection schedule should be designed so that additional ICP
products are collected as part of the normal collection timetable and mostly in the sameoutlets as all other CPI products. This will avoid ICP products being left to the end of themonth and possibly not being collected if there are time overruns. It should also reduce
travelling and time costs for the NSO as price collectors will not need to return to an outlet
or area to collect ICP data.

6.2.2 Support materials and training
Key recommendation: All field staff should be trained on product specifications, variables, andchanges to data collection forms.
Having highly trained data collectors reduces the incidence of error and other issues that can affectquality. It considered best practice to provide data collectors with refresher training each year.
There are two important elements of training to consider:•First, fieldworkers must understand the new specifications to be recorded and the ways inwhich they are to do this. They must also know how to complete any fields in the pricecollection form that have been changed to accommodate ICP needs.

•

Second, they must have adequate knowledge of the overlap and additional ICP products thatshould be collected. This knowledge includes the restrictions on each product such as brandor size.

A product catalogue is a useful tool for price collectors. This is a visual aid containing a photographic
example of the type of item and a description by way of completed variables that appear on theprice collection form. The product restrictions required for the ICP can also be included in the
product catalogue.

286.3 Processing and editing
6.3.1 Capture data
Key recommendation: Data required for the ICP should be stored in the same database as the CPI
and other price data.
All prices should be captured and stored in single database. In cases where CPI data are collectedusing mobile devices, data collection and initial editing are combined.
This means that adaptations will need to be made to the CPI system to allow collection of theadditional specifications as discussed above. Because of the precision required in defining products
for the ICP and ensuring that identical products are compared for the calculation of average prices,
the fields should be populated by predetermined drop down lists. This eliminates any variation inspelling or punctuation that can impede the speed and efficiency of the editing processes.

6.3.2 Validate and edit data
The aim of editing is to ensure that the data are fit for purpose for the next phase of the process. All
CPI programs should have quality assurance processes in place to detect errors made in pricecollection and data entry. Applying additional requirements and procedures used for the ICP may
enhance the validation methods used the CPI.
The first check should be to ensure that the varieties selected for pricing conform to the restrictions
imposed by the ICP. This should initially be done by field supervisors; and then again by thestatisticians responsible for data editing and quality assurance. This phase is essential when an item
is being selected or priced for the first time. This check should also confirm that the characteristics
have been correctly recorded.
The initial editing phases of the data should include checking the detailed specifications as well as
the prices. This would identify any errors or omissions from either the data collection or data entry.
Where possible, corrections should be made to the data – so long as these are not making
unsubstantiated assumptions about the missing information. For example, editing could easily fix aspelling error or a unit of measure but perhaps not the actual weight or quantity. Entries that cannot
be corrected appropriately should be excluded from further processes. A log of all changes should
be maintained for later analysis. This can point to areas in which staff may require further training.
Calculation of average prices, which is fundamental to PPP compilation, may also be useful for CPI
editing. Identifying outliers among prices constituting an average can highlight products that do notcomply with specifications or are of different quality than other purchases.
The final processing phase will only apply to those price observations that are to be used in the ICP
calculations. The ICP editing system ensures compliance to strict definitions and standardization in
an iterative fashion.

29

Following the recommendations above will substantially reduce the time needed for this final editing
process. It should also limit the need for regional meetings to conduct editing activities – further
reducing funding requirements.

7. Scanner data and web-scraped prices
Increasingly national CPIs are using alternative sources of data to enhance traditional pricecollection. These include (i) scanner data, which provide a particular type of transaction data thatuses the barcode, that is, the Global Trade Item Number (GTIN), for presenting aggregatedinformation on the sales of products, and (ii) web-scraped data, which reflect the offer or askingprice for a product or service.
Scanner data on transactions sourced from retailers hold significant promise for CPIs as the
granularity of the data provides more information than the recording of a price once a month.
Furthermore, the marginal cost of obtaining additional prices is negligible or zero in many countries,
which will significantly help to reduce the resource requirements of data collection. However, it
should be noted that scanner data must be purchased in some countries and it can be very costly.
Scanner data not only supplies the price paid for a product, but also the quantity purchased in eachtransaction. Scanner data typically include a description of each variety together with the barcode.
If the description includes all the variables required for ICP purposes, these data should be used. If
the description is not packaged into discrete variables then some programming will be required toseparate and extract the brand and size, etc. This allows the calculation of price indices with realtime weights using a superlative index formula. The availability of scanner data has implications forthe aggregation procedure used at the elementary or basic-heading level. The ICP currently uses thecountry-product-dummy (CPD) method without any actual weights, aside from the weights basedon importance indicators. The availability of quantity data with price observations makes it possible
to introduce quantity or expenditure-based weights at the basic-heading level. Consequently, theuse of scanner data could eliminate the need to use importance indicators for products as proper
quantity and expenditure-based weights are available at the product level. In addition, weights fromscanner data could inform the development of item lists.
Web scraping is a technique used to harvest data from online sources, such as the websites of
traditional outlets, web-based outlets, and online marketplaces or platforms. Because therepresentativity of the varieties are not available on the website, web scraping is a supplement to,rather than a replacement for, manual price collection. Web-scraped data should be suitable for use
in the ICP as the characteristics of the variety can be recorded in discrete fields by the program.
However, representativity of web-scraped prices should still be assessed for ICP use.

30

8. Housing and dwelling services
Treatment of housing is an important element in the compilation of CPIs as well as in the compilationof PPPs within the ICP, given associated consumption weights. Part D of Chapter 11 of the CPI Manual
discusses the treatment of housing in CPIs. Chapter 12 of “Measuring the Real Size of the World
Economy: The Framework, Methodology, and Results of the International Comparison Program(ICP)”,23 discusses the problems with compiling PPPs for housing. The problems encountered in thecontext of the CPI and ICP are similar and, therefore there are significant benefits for countries tointegrate and develop a unified approach to making temporal and cross-country comparisons ofhousing costs.
The main problem arises in measuring costs for owner-occupied housing. The CPI Manual discusses
various options including: (i) acquisitions approach; (ii) the payments approach; and (iii) the user cost
approaches. In the case of CPIs, a measure of price change for dwellings would be based on rents foran appropriately designed stratified sample of properties, along with weights reflecting theimportance of each stratum.
The methodology recommended for the compilation of PPPs for housing within the ICP is similar to
the rental equivalence approach. The recently released report on the ICP 2017 cycle by the World
Bank, “Purchasing Power Parities and the Size of World Economies: Results from the 2017
International Comparison Program”24 (World Bank, 2020, page 80) explains the approach currently
recommended as:“…Under the rental approach, rental prices are collected for similar dwelling types in eacheconomy. This method has been found to work well in economies in which the dwellings
actually rented are representative of the stock of dwellings as a whole and where statisticalagencies collect information on rents paid for the different kinds of dwellings that are rented
in most parts of the economy.”

9. Conclusion and next steps
Harmonization and integration of CPI and ICP related activities have long been recognized as criticalto the sustainability of ICP activities. At the recommendation of the Friends of the Chair of the United
Nations Statistical Commission, the ICP Global Office has identified this as a priority topic for the ICP
Research Agenda.25 Results presented in Section 5 illustrate that countries involved in thecompilation of PPPs as a part of the ICP have already recognized the importance of CPI-ICP
integration and have been implementing strategies to promote such integration at varying levels.

23

https://www.worldbank.org/en/programs/icp/brief/2011-icp-book24

https://openknowledge.worldbank.org/bitstream/handle/10986/33623/9781464815300.pdf

25

https://www.worldbank.org/en/programs/icp#731

An attempt is made in this guide to identify key areas for CPI-ICP integration and to articulate a setof recommendations designed to promote integration and to realize the potential synergies arisingout of CPI-ICP integration. The guide employs an approach based on the statistical value chain withparticular focus on the design, price collection and, ultimately, on the processing and validation ofdata used in PPP compilation. In addition, several case studies on CPI-ICP integration are appended
to this guide.

32

References
Chance (1966). A note on the origins of index numbers. The Review of Economics and Statistics. Vol.
48, No. 1. (February 1966), pp. 108–10
Gilbert, M., and I. Kravis. 1954. An International Comparison of National Products and the Purchasing
Power of Currencies: A Study of the United States, the United Kingdom, France, Germany, and Italy.
Paris: Organisation for Economic Cooperation and Development.
International Comparison Program (2021), A Guide to the Compilation of Subnational Purchasing
Power Parities (PPPs), World Bank, Washington D.C.
https://thedocs.worldbank.org/en/doc/5064f2288436664bc8f9811c8a5b8c550050022021/original/Guide-Subnational-PPPs.pdf
IMF et al (2020), Consumer price index manual: concepts and methods, Washington, DC,
International Labour Office. | International Monetary Fund. | Organisation for Economic Cooperation and Development. | European Union. | United Nations. | World BankWorld Bank (2013), Measuring the size of the World Economy: The Framework, Methodology and
Results of the International Comparison Program (ICP), Washington DC, World Bank.
World Bank (2015), Operational Guidelines and Procedures for Measuring the Real Size of the World
Economy 2011 International Comparison Program, Washington DC, World Bank.
World Bank (2020), Report Purchasing Power Parities and the Size of World Economies: Results fromthe 2017 International Comparison Program, Washington DC, World Bank.

33

Appendix A: CPI-ICP integration questionnaire
1. Respondent details
Country
Name of institution
Name:
Person completingquestionnaire

Position held and Department:
E-mail:Is the collection of prices for ICP carried out by the same organisational
unit as price collection for CPI?

Yes

No

2. Price collection
Please indicate the approximate number of locations and outlets and price observations that arecollected, on average, each month/quarter.
Number of
Locations

Number ofOutlets

Frequency of pricecollection (monthly orquarterly?)

Number of
priceobservations

CPI
ICP

Number of products in CPI and ICP basket
Total number of products
in CPI basket

Number of ICP products in
CPI basket (overlap)

Number of ICP products not
in CPI basket (extra
collection)

Price collection
Do you use the same form/questionnaire/hand-held device for CPI and ICP
price collection?
Number of price collection staff (full time equivalent) for CPI

34

Yes

No

Number of price collection staff (full time equivalent) for ICP

Are the price collection staff for ICP a subset of the
CPI price collection staff?

Yes

No

Other: Someare andsome notDo staff working on ICP receive training on the ICP requirements?

Yes

No

Is there a product catalogue/photobank to assist field workers with ICP
collection?

Yes

No

Is price collection for ICP carried out at the same time as the
CPI price collection?

No

Only onoverlap
products

Does your NSO receive additional funding to carry out the ICP
collection?

Yes

No

Do staff members of the NSO receive an additional allowance to carryout the ICP collection?

Yes

No

Yes

Please indicate the geographic coverage of your country’s ICP household consumption survey.
Please select the statement that most closely applies.
Coverage is national (includes urban and rural
areas, as per the national definition of“urban” and “rural”)
Coverage is urban only (includes the capitalcity, plus other main cities and metropolitan
areas, but excludes surrounding rural areas)
Coverage is capital city only (includes only thecapital city – note: can include surrounding
urban areas)
Other geographical coverage (please specify)

Please indicate whether spatial adjustment factors (SAFs), used to convert urban average prices and/or capital city average prices to national average prices, were produced for your country’s ICP
household consumption survey.
35

Please check all that apply.
SAFs were produced for individual products
SAFs were produced for lower level
aggregations such as ICP basic headings
SAFs were produced for higher level
aggregations such as ICP classes, groups, orcategories
Other, please explain
Not applicable

3. Data processing
Are ICP data captured in the same computer system as the
CPI data?

Yes

No

Only onoverlap
products

Do you use the same editing/control procedures to ensure
quality on CPI and ICP data?

Yes

No

Only onoverlap
products

Do ICP validation procedures help improve yourprocessing of CPI data?

Not
usefulSomewhat Very
usefuluseful36

4. GeneralDo you as an organisation have plans in place to implement changes that will
improve the integration of CPI and ICP price collection and processes in thenear future?

Yes

No

Do you have suggestions on how to improve the integration of CPI and ICP price collection andprocessing?

37

Appendix B: Case studies

CASE STUDIES FROM INDIA, ISRAEL, UNITED KINGDOM and SOUTH AFRICA

38

CPI/ICP case study: India
1. Introduction
India has participated in all ICP rounds, except for 1993. Most of the participating economies,
including India, especially in the Asia and the Pacific Region, have been conducting fresh pricecollection surveys for a basket of goods and services in a number of selected markets in thereference years of respective ICP rounds. Conducting separate surveys requires deploymentof additional manpower and financial resources, which results in delay in respect of many
countries and the quality of national average prices for the reference period is consequently
affected.
Now that the ICP has become a permanent element of global statistical program, it has become
necessary to integrate ICP with CPI. Most of the countries produce a CPI (monthly or quarterly) for
which they collect prices of household consumable items. In the ICP household items account for
more than 80% of the number of items or products (not 80% of GDP expenditure). Therefore, theaforementioned synergy can reduce significant burden of survey for ICP and energy of the National
Statistics Office would more be devoted towards validation of national average prices.

2. Dimensions of ICP-CPI synergy
There are two dimensions of ICP-CPI integration: (a) integration of sample design for selectionof markets and outlets, and deployment of same manpower for conducting both the surveys of
ICP and CPI; and (b) integration of baskets of household consumable goods and services. The f irst
dimension ensures good coverage of markets, at reduced cost, for estimating robust national
average prices whereas the second dimension emphasizes on regular flow of quality data andprovides detailed information on important and less important items of the ICP basket.
2.1

Integration of sample design

Integration of sample design is easy to implement by any country. The entire set of markets of the
CPI or a subset thereof may be taken and the same manpower may be deployed for price collection
for the ICP. If these markets do not represent t h e country as a whole then the average of theprices are poor estimates of national average prices of ICP basket.
In the case of India, the CPI is compiled using prices from 2,295 markets (1,181 rural covering almost
all districts and 1,114 urban markets of 310 towns) of the country. There is an adequate
representation of each part of the country. Therefore, a subset of the markets (taking representationfrom each region) is sufficient to give good estimates of national average prices.
2.2

Integration of baskets

Implementation of the second dimension is a major challenge due to weak commonality between
Structured Product Descriptions (SPDs) of the ICP and those of CPI baskets. The ICP prioritizes
comparability over representativity whereas the CPI hinges on representativeness. Enhancing
comparability compromises on representativeness, which affect the quality of CPI and vice versa.
Therefore, a model is required to be developed for integrating both the baskets without affecting
the quality of these indicators.

39

3. Methodology proposed to be adopted for integration in India
3.1

Integration of sample design and manpower deployment

India used this dimension of convergence in earlier rounds of ICP, especially in 2011 and 2017, by
taking a subset of the CPI markets. For ICP 2017, prices were collected from 577 urban and 320
rural markets.
For t h e ICP, subsets of t h e aforementioned rural and urban markets (all markets of those large
NSS Regions which have 12 or more markets in each rural and urban sector, ensuring at least
one NSS Region in each State for giving nation-wide representation) would be taken in the ruraland urban samples of markets for conducting the price survey of household items. A total of 947
rural and 946 urban markets are being covered for the ICP price survey.
The outlets identified for the CPI are to be used for collection of prices for ICP items in the
aforementioned rural and urban markets. It is important to mention here that shops selected for
CPI items are the most popular outlets; therefore, there is a fair chance that products ofdifferent specifications of ICP are available in these shops.
3.2

Integration of ICP and CPI baskets

As mentioned before, the major challenge in establishing synergy between both the baskets is poorcommonality. India is a vast country having varied consumption patterns and preferences of goods
and services within short distances. Therefore, a large sample size is required for the compilationof the CPI to capture all types of consumption behavior. SPDs of CPI items are fixed market-wise sothat the prices of most representative products are collected during the survey. As a result, SPDs of
items differ from market to market. Even if some of the SPDs of the CPI basket are identical to thoseof the ICP basket, this would not hold true for all the markets of all different States. Therefore,
those CPI prices may not be used to compute national average prices.
While examining the ICP product catalogue of Asia and the Pacific, it has been found that the SPDsof fruits and vegetables may be considered common to those of the CPI in a fairly large number ofmarkets. But in the case of non-CPI items in the ICP, conducting a parallel survey is required.
There are about 930 household items/products in the ICP. Out of these, about 40 items fall in the
category of fruits and vegetables. Even after excluding these items, there are about 890 items tobe added to the list of CPI items, if their prices are to be collected along with the products containedin the CPI basket. Conducting a survey for such a long list is practically not feasible.
In view of this, it is proposed that prices of fruits and vegetables collected in each market for
CPI would be used for ICP baskets also; and for the remaining household items, the ICP
basket would be divided into 12 parts. One part would be allotted to one market of that NSS
Region of the respective sector. Here, it is important to reiterate that these are large NSS Regions
having 12 or more markets in each sector. Thus, there would be a n addition of only 890/12 =74 items to the CPI basket of a particular market of the respective NSS Region. If the number ofmarkets in a NSS Region is more than 12 then the items falling in the category of Food, Clothing
and Footwear (about 415 to 420 items) would be distributed in the remaining markets (other than
the aforementioned 12) of the respective sector. For instance, there are 23 markets in the ruralsector of an NSS Region of Chattisgarh. In this case, 12 different sets of 74 items would be40

added to t h e CPI baskets of 12 markets. Then in the remaining 11 markets, 11 different setsof 38 items (420/11 = 38) of Food, Clothing, and Footwear would be added to the CPI baskets.
These ICP items would be clearly classified as ICP items in the schedule of price collection anddata entry software in order to avoid confusion.
There is another option of splitting the ICP basket over time; for instance, one-third of the basket
is surveyed in the first, second and third months of a quarter. Durable items (other than Food,
Clothing, and Footwear) may be even surveyed every six months. A similar approach was followedby India in ICP 2011 and 2017. But the strategy mentioned above has been found more appropriate
because of following reasons:•the proposed method adds only 75 items to the CPI basket of a particular market whereasanother approach includes a greater number of items; and•prices are collected every month, which captures month to month fluctuations in prices
without compromising the coverage of space.

Adoption of this approach neither affects the representativeness of CPI baskets nor compromiseson the spatial comparability of ICP items.
Increasing the size of the basket leads to reluctance on the part of a shopkeeper, which results
in non-response and not reporting the actual transaction prices. Therefore, it is necessary thatthe basket should of optimum size. In order to compensate the additional load of about 75 ICP
items on CPI basket, a methodology has been worked out to reduce the size of CPI basket.
The experience of the existing series of CPI says that names of the item containing t h e word‘other’ under each sub-group poses a problem in price collection since the SPD is not welldefined. Therefore, such items would be removed from the CPI basket of the revised series, andtheir weights, if any, be pro-rata distributed on the remaining items of the respective sub-groups.
This strategy is expected to exclude about 70 to 80 items in the baskets of CPI of a particularmarket, which gives sufficient scope to add 75 ICP items to CPI basket without putting an additionalburden on the data collectors.
The collected ICP prices would pass through scrutiny checks every month along with CPI data
and validated data would be stored for computation of national average prices as and whenrequired by the Asian Development Bank (ADB), the Regional Implementing Agency for ICP Asiaand the Pacific.
3.3

Methodology for preparation of different sets of 75 ICP items

While preparing sets of 75 ICP items/products, it would be ensured that all categories of COICOP
are proportionately represented in each set and all items of a particular basic heading would besurveyed together in the same market. Further, deviations of SPDs of CPI items from that of ICP
items would be examined marketwise. If there is a difference in terms of variety, brand, quality,
model etc. then items of both the baskets would be considered uncommon. On the other hand,
if deviation occurs due to a difference in unit and quantity, then items of both the baskets may
be considered as approximately common. If such commonality is found, in respect of a41

particular product and in majority of the markets, then that item would be classified as animportant item. This analysis would help in providing information on ‘Important’ and ‘Less
important’ ICP items for assigning suitable weights by the PPP compiling agency.

42

CPI/ICP case study: Israel
1. Introduction
The Central Bureau of Statistics in Israel (ICBS) has participated in the International Comparison
Project (ICP) on a regular basis since 1996.
The global offices in charge of conducting the ICP are introducing new and improved features
to the program on a continuous basis. These included in the 2005 round:•A new Structured Product Description (SPD) approach that has improved the scope andquality of regional and global lists of specifications.

•

A comprehensive Handbook and a set of operational guidelines that serve both asreference material and step-by-step guidance for data collection and compilation.

•

An integrated software system, the Tool Pack, that has been developed for standardized
data collection, verification and processing, and capacity building.

•

Improvements in construction and equipment surveys, aggregation of PPPs at the basicheading level, linking of regional results to estimate robust global PPPs, and estimationof poverty-specific PPPs.

The participating countries like Israel have benefited from some of these changes. However, we
remain responsible for the tedious task of price collection, a task performed without receiving
special resources as the PPP does not carry the same “clout” as the national CPI. Therefore,
we seek synergies in the process and search fo r ways to utilize the resources already allocatedto the routine CPI program in order to keep to PPP standards.

2. Issues in price collection for PPP
Price collection is the responsibility of the participating countries. Before they can begin pricecollection, they must complete a number of tasks. These involve: selecting and contacting theoutlets to be visited by price collectors; preparing the pricing materials and supplementary
documentation for price collectors (including, if necessary, the translation of survey guidelines
and product specifications into the national language); identifying which specifications on the
final group product list are to be priced and, in the case of generic specifications, which brands
are to be priced; and holding a meeting with price collectors to clarify issues such as how many
items per basic heading, how many price observations per item, etc.
The selection of outlets is of particular importance because of the effect it will have on the
average prices of the products to be surveyed. Different products have different distribution
profiles. Some products are sold mostly in supermarkets; other products are sold mainly in
specialist shops. Prices for the same product can vary from outlet type to outlet type because it
is being sold under varying conditions.
A common starting point for the selection of outlets is the sample of outlets used for the CPI,
43

but it is only a starting point. The product lists for the Eurostat and OECD price surveys are larger
than CPI lists, specifying products not included in the CPI. The CPI sample may not be ideal tocollect reliable prices for these products because the selection of outlets by type is not in
proportion to the volume of their sales of the products. In this case, it is necessary to augment
the CPI sample with additional outlets. At the same time, because the resources available for the
price surveys are limited, the CPI sample may be too large and has to be reduced.

3. The present work processes
The Division of Consumer Prices at ICBS assumed responsibility for PPP compilation as of January
2004. In practice we collect prices for the 3,000 goods and services defined in the programthat serve as the basis for computing the PPPs and international comparability.
Price collection is conducted via paper questionnaires that are sent to us by OECD. The overall
work process that was established at initiation was as follows:•

The dataset is received from OECD and sent to the price coordinator for “deciphering”.

•

A weekly timetable for price collection is defined.

•

Price data are collected: price (standardized to definition), date of collection. Also punched
into a different dataset are the details of the outlet and specification of any problems
encountered in the collection process.

•An editing and logical check process is applied. Data may be supplemented from the Internet
or CPI data. The PPP coordinator sends the edited data to OECD.

•

The data are sent back from OECD to the PPP coordinator for questions, clarifications, etc.

•

Additional checks are conducted in the field, when deemed necessary.

This complex process has led to several problems that were defined in order to look for ways toimprove the collection and editing stages of the PPP. These problems include:•

Refusal to cooperate – in contrast to the CPI where there is a legal obligation to participate,
the PPP program required “lobbying” on our part in order to ensure participation. This maylead to non-response errors that arise from the failure to obtain required information in afull and timely manner from all the units selected in the sample. If the prices of thenonresponding outlets differ from those of the responding outlets, the results of the pricesurvey will be biased.

•

Definition discrepancies – between goods that are defined in the questionnaire that may beirrelevant or non-existent in the local market. The problem is acute in electronic appliances,clothing, and footwear.

•

Judgmental sampling – as there is no one single sampling frame for a CPI, non-probability
sampling or “second degree judging” in a PPP program is conducted at times by the pricecollector.
44

•Although empirical results of research undertaken by Statistics Netherlands for the CPI
nevertheless show that non-probability selection methods do not necessarily perform
worse, in terms of the mean square error, than probability sampling techniques – one would
assume that this is due to the sound methodology of CPI compilation at all stages.

•

Sales – the handling of sales prices in our PPP exercise may be dubious due to the sporadic
nature of the price collection.

•

Seasonality – the problems of seasonality are magnified in a PPP price collection that is not
spread throughout the calendar year for all questionnaires.

•

Using price indices for adjustment of PPP prices over time – this process assumes a good
match between the selected CPI and the good or service from the PPP exercise. At times thismay be a strong assumption.

•“Made in China” – the local market is flooded with electronic appliances that are dominant
in household purchases and are of different technology and prices than those that are
defined in the questionnaire.

•

Many prices that are needed for the PPP exercise (down to the exact SPD) were alreadybeing collected in the CPI program. Furthermore, at times, prices for PPPs were collected instores that were located “next door” to the CPI outlets.

•

The price indices computed for adjustment of prices to specific time periods for OECD weredone by hand and in non-economical fashion.

4. Enhancing the Synergies between CPI and PPP
Many of the problems described in the former section stemmed from the “disconnect” between
the two programs. Realizing that the CPI program can be utilized in many ways in favor of PPP
we initiated a program in the 2005 round that enhances the synergies between the programsin three areas: (i) price collection, (ii) price methodology, and (iii) CPI technology principles.
4.1

Utilizing CPI price collection to enhance PPP price data:

Many of the goods and services that are collected in the CPI framework are also requiredfor PPPs (especially household expenditure). The following procedures were introduced:•

Using the prices of the following services for PPP: electricity, gas, water, public
transportation, health insurance, car insurance, recreation, and several other personal
services are taken directly from the CPI without requiring additional resources.

•

The prices of goods in the CPI can be used for supplemental information and logicalchecks.

•Fresh fruit and vegetables: average annual prices are computed based on the monthly
CPI collection. This alleviates the problem of seasonality mentioned above.
45

•The selection process of items in the CPI is based on the most popular item as
determined by the proprietor. This is also the item required in many cases for the PPP
exercise. We now combine between the two.

•

Increasing the P P P sample size based on the CPI outlets. In cases where the PPP sample
is not large enough, we can use CPI data to compute an average price with the PPP data.

•

Refusals have been brought to the minimum due to the fact that collecting the CPI data
for PPP purposes has elevated the legal binding of these data under the statisticalordinance. In addition, the PPP price collector has gained experience and established cordial
relations with the shopkeepers.

4.2

Combining price methodology of CPI and PPP in the fieldwork:•

Sampling procedures for the CPI have been expanded to allow for inclusion of the PPP
exercise.

•Use of administrative data for the CPI (like housing, health records, etc.) in the PPP exercise.

•

CPI methodology requires selection of the popular items. PPP methodology requires a mixof representativity in local markets and international comparability – two forces thatare contradictory. We now try to apply this method in the CPI in order to be able to serveboth programs.

•

Collect regular p r i c e s and sales prices for PPP and send this data to the OECD (whenrequired) for their decision.

•

Editing and logical checks are performed in the CPI unit under the supervision of the Headof Division and using the same methodology as in CPI.

4.3

Similar technological principles for PPP:

A system which automatically classifies PPP and CPI into identical categories was developed.
This enables current updates of the indices needed for the PPP exercise by OECD. The CPI basket
is updated every two years and the computer system accordingly. The system was developed inthree stages:•

The consumption items that were identical for both programs were selected.

•

Special aggregates were built for the requirements of the PPP exercise.

•

The datasets were classified according to the same international classification.

An initiation document for developing a total PPP system within the framework of CPI is now inprocess.

46

CPI/ICP case study: United Kingdom26
1. Introduction and historical background
The role of the UK as it currently stands is that as part of the European Union. There is uncertainty
over the future role and membership of the UK in the EU due to Brexit. The UK is currently(primarily) funded by Eurostat to deliver the outputs of the European Comparison Programme27(ECP). However, there is a risk that if the UK leaves with a ‘no deal exit’ funding will be withdrawn.
It is therefore imperative that, going forward, the UK utilizes its current resources, including pricedata, to its full effect. This includes using CPI data where possible, both for use in PPP calculations
as well as for quality assurances purposes and taking advantage of the challenging and significant
work that ONS is currently undertaking regarding using Alternative Data Sources (ADS).
The main area of integration in the shorter term will be the use of Alternative Data Sources such as
web-scraped and scanner data for both CPI and PPP outputs and not one necessarily of using thesame systems and processes to calculate the different outputs. This is because currently thesystems and processes which underpin the PPP calculations are developed and supported byEurostat, whilst the CPI is imbedded in systems and processes developed and supported internally,in ONS; therefore, minimum synergy exists.

2. Main objectives and motivation
The ECP basket has more than 2,500 items whereas the national UK CPI has approximately 750
items. The UK CPI basket comprises representative goods of the UK domestic market whereas the
ECP/ICP basket comprises both non-representative and representative items which are importantin all markets of the two programs (both ECP and ICP). Countries are expected to collect both nonrepresentative and representative items due to the way the indexes are calculated. This
fundamental difference in the baskets is the main barrier to the integration of these two outputs.
The main objective or motivation of integrating CPI and ECP/ICP must be resources, both cost andefficiency. The aim is that each price collected could be used in several outputs. Currently in the UK
prices are collected separately for the UK CPI and the ECP/ICP. There are a few exceptions: petrol
and energy prices.
Outside of the calculation of the Spatial Adjustment Factors (SAFs), (where more than 250,000 CPI
prices were used for PPP purposes) there is very limited sharing or integration of that data that areused by either party. With the exception of petrol and energy and for validation purposes (i.e.
ensuring that the PPP movements align to the movement in the CPI), PPPs do not use CPI data andvice versa. However, the use of CPI data for the ECP SAF project demonstrates that CPI data aresuitable for PPP purposes, albeit not across all products. However, with increasing demand on
26

This note was written prior to the withdrawal of the United Kingdom from the European Union on January
31, 2020, and thus refers to uncertainty over the impact of BREXIT.
27

The ECP is part of the Eurostat-OECD PPP comparison.

47

resources and an appetite to ‘collect one price that could be used for all’, there is a need to collect
one price and utilize that price data fully.

3. Initiatives for CPI-ICP integration
3.1 Alternative Data Sources
Alternative Data Sources (ADS) have been growing in importance over the last few years within
ONS as a means of collecting price data as an alternative to the more traditional methods ofcollecting, such as out in the ‘field’, by telephone, or via the internet.
a) Scanner data
Scanner data are data collected by retailers at the point of sale. It will provide us with information
on what products are sold, how many were sold, and at what price, giving ONS significantly more
information about prices, but also what expenditure weight each item should have within our
virtual shopping basket. Currently for PPPs the lowest level at which expenditure weights areavailable are at the basic-heading level. ONS is currently engaging with key retailers in the UK
obtaining scanner data on items ranging from food through to alcoholic beverages and personal
items.
b) Web-scraped data
Web-scraped price data are collected from retailers’ websites and can provide a wealth ofadditional product information about online prices, such as product descriptions. For example, as
well as obtaining the price of a laptop, ONS will collect information such as the laptop’s RAM andprocessor speed, which help us to ensure that the products we are comparing are the same quality
over time.
In the ECP consumer survey of House and Garden, the UK PPP team embraced the new technology
and used web-scraped data for 15 electrical items. Python was used to run the coding and scrape
the website overnight (as this had been agreed in advance with the retailer). This was repeatedseveral times during the four-week collection period which allowed for not only a spatialcomparison with other countries but also a temporal comparison for the UK.
3.2 Areas where integration is achieved
From a UK perspective, and due to the national importance of the CPI, it is highly unlikely that ECP
data will be used for CPI but rather, CPI prices will be used for ECP/ICP purposes. This is primarily
caused by the detailed specifications required in the ECP collection in comparison to broader CPI
item specifications.
The example below shows the difference between the ECP and the CPI specification and the levelof detail required for the ECP collection. For example, in the UK CPI a women’s shoe item allows acollector to price for any high heel shoe regardless of brand or material composition if it’s the sameor comparable to the previous month’s CPI item. For PPP purposes, the shoe will have a moredetailed specification, including details such as brand level, material composition and the heel48

height. Due to these differences in the specifications, it is not possible to use the average prices
taken from the CPI collection, as it is unknown if observations would meet the PPP specification
requirements.
Table 1: Item specifications for ladies’ shoes
ECP item specification

CPI item specification

Ladies conventional court shoes, WKB-H

Women’s shoes, high heels over 1.5”/4cm

Well Known brand – High

Material – can be man-made, leather, speciality
fabric, upper and soles

Type - Court shoe
Without decoration

Fastening can be slip on, buckle, button,
lace

Upper Material – Leather
Sole Material – leather or synthetic
Heel height – approx. 7cm
Specify brand and sole

Therefore, for the regular ECP/ICP consumer surveys that the UK undertakes, very little CPI data isused, rather it is used for quality assurance purposes.
That said, however, the UK was successful in using CPI prices in the PPP calculations when it
undertook a nationwide regional price survey to calculate correction coefficients (Spatial
Adjustment Factors (SAFs)). These SAFs are used to adjust the capital average prices (in the case of
the UK, London prices) of those observations collected as part of the ECP consumer goods andservices surveys. Due to resource constrains, prices for the ECP are only observed in the capital ofeach of the member states. However, the requirement is to calculate national average prices andnot capital prices and therefore every six years a project is undertaken whereby prices are collectedfrom regions within the UK. The resultant correction coefficient is then applied to the capital
average prices, at basic- heading level, along with Temporal Adjustment Factors (TAFs) to arrive ataverage national prices.
3.3 Sectors where CPI data are appropriate to use for ECP/ICP
As part of the SAF project it was necessary to use CPI data wherever possible, to reduce costs andimprove efficiency. Potentially, there are more than 2,500 items in the ECP basket of consumer
goods and services, and analysis was conducted as to how many of the 750 CPI items could
potentially be used to represent these 2,500 items. Many items had synergies between the twothus allowing integration of the CPI and ECP. Where there was synergy in the item specifications,this ensured that resources in the field could be utilized effectively and could be used on those
items where either CPI data did not exist or where the specifications were difficult or narrow and
CPI data could not be used.
49

Additionally, a concerted effort to understand where national pricing exists ensured that a targeted
data collection process could take place. Approximately 277,500 prices were utilized from the UK
CPI dataset.
Food items in general tend to have more synergy and therefore integration is easier to obtain,
hence the high number of CPI items (>50%) used for ECP purposes for the aggregate of Food,
Beverages, Alcohol, and Tobacco. Refer to the item below, ‘fresh watermelon’, where the CPI data
can be used for ECP/ICP purposes as the specification is virtually identical.
Table 2: Item specifications for fresh watermelon
ECP item specification

CPI item specification

1

Fresh watermelon. 1KG

Kilogram
Fresh watermelon
Sold loose
MediumGood quality; intact (unbroken, unmarked) skinExclude: Fair trade fruits, organic fruits
State: Whether "in-season" or "out-of-season"
Ref quantity:1kilogram

However, this is not the case for many of the items in the ECP/ICP Furniture and Health survey due
to the very detailed and tight ECP/ICP specification of furniture items. Comparability acrossfurniture items is difficult to achieve and is always a challenge. Therefore, the ECP/ICP specificationmust be very detailed and ‘closed’ to achieve harmonization across all participating countries inboth programs. For CPI, however, the key is to collect the same item month-on-month regardlessif the item is WKB-L or WKB-H or indeed brandless. Therefore, no CPI prices could be used for thisaggregate. For reasons of comparability across the ECP/ICP items, it would not be possible to havea less descriptive item for furniture; the ethos of both the ECP and the ICP is comparability andtaking such an action would put this at risk. The only alternative is to make the CPI item more
descriptive. However, with resources being a challenge for many NSIs and with the CPI taking
priority this will be an impossible task using the current method of collection, either out in the field,
via the internet or telephoning. However, this could be achieved via the use of alternative data
sources such as web-scraped and scanner data.
Refer to furniture item below which demonstrates the reasons why many CPI items cannot be used
for ECP/ICP purposes50

Table 3: Item specifications for spring mattress
ECP item specification

CPI item specification

Spring mattress, single spring system, without Mattress, 3ft/90cm, interior sprung
comfort zones

Brandless1 Piece
Spring system, single, without comfort zones
Cover: cotton or mixed material, Padding: foam
plastic, liner: felt (comfort material)
90 cm
17 - 20 cm (thickness)
200 cm or national standard
Delivery charge
Mattresses with pocket springs, mattresses
with integrated wooden frame, mattress with
comfort zones
Label, if any; dimensions

3.4 Survey framework – coverage (cities, rural-urban, provinces-regions)
Synergy already exists to some degree between CPI and ECP/ICP in the survey framework in termsof:a) Locations
For the locations, there are some, albeit limited, synergies between the sampled locations of the
CPI and ECP/ICP for the reasons listed below.
For the ECP/ICP, prices are collected in the capital city of the UK, from ten established locations
based primarily on ‘hot spot’ analysis and the knowledge and experience of the PPP price collectors.
It would be too resource intensive to conduct a nationwide survey for each PPP consumer survey.
However, the requirement for ECP/ICP is to produce annual national average prices and therefore
correction coefficients in the form of Spatial Adjustment Factor (SAFs) are applied to the city prices
to arrive at annual national averages.
Prices for the UK CPI are collected nationally from approximately 141 locations across the UK.
51

Location selection takes place separately within each region in the UK, using PPS (Probability
Proportional to Size) systematic sampling with a size measure of the number of employees in theretail sector. The number of locations in each region is determined as the proportion of national
expenditure that takes place in that region, multiplied by the total number of locations to be visitednationally.
That said, however, there are several locations within the city of London that are sampled for both
the CPI and ECP/ICP.
b) Outlets (both retailers and service providers)
Synergies exist for location sampling for both the CPI and ECP/ICP where outlets are selected by
PPS sampling or, if this is not necessary, then by simple random sampling (SRS). The actual sample
selected for outlets for both the CPI and ECP/ICP is primarily the same, except for possibly the small
independent outlets where random sampling is used or sometimes the outlet is sampled based on
the knowledge of the price collector and the relationship that they have built with the owner of theshop.
c) Items
Price data are collected for around 700 items for use in the UK CPI compared with around 2,500
items across the ECP consumer surveys. There is therefore an immediate disadvantage in that the
sample of CPI items is far smaller than the ECP sample which in turn makes the scope and extent ofsynergy of the items more difficult. It is also extenuated by the fact that the ECP/ICP baskets include
both representative and non-representative items. The national CPI basket, as you would expect,consists of representative items.
However, there is scope for NSIs to influence the content of the basket of goods and services. The
current practice for updating the ECP basket is that prior to any new ECP survey, proposals are put
forward by participating countries for new items. This is to allow for changes in consumer spend in
the intervening three years before each survey.
For the CPI, for most goods, the selection of items within outlets is purposive (the exception being
some goods, where local probability sampling is applied). In each sampled outlet, collectors in thefield choose one variety representative of what consumers would typically purchase in that areafrom all items matching the specification of each item to be priced in that outlet. To facilitate this,they ask the retailer what the most popular brands are and those stocked regularly; it is imperativethat the same product is priced each month. This is a very different approach to the one used for
ECP/ICP items where strict product specifications are adhered to, to ensure the integrity of the
produced statistics; this is a major challenge (already covered in Section 3.1 above) and limits
possible synergies and therefore the integration of CPI and ECP/ICP items.
3.5 Price surveys
Currently different price surveys are undertaken for the CPI and ECP/ICP and it is envisaged that
this will continue until there is more synergy between them that will allow for integration of somepart of the collection and processes.
52a) Consumer price index
The CPI undertakes a monthly price collection out in the field across 141 locations randomly
selected covering approximately 700 items within the UK over a period of 7-10 days depending on
the item(s) in question. This survey is primarily conducted by a third-party contractor. Around 300
part time collectors conduct this collection. Ten field auditors employed by ONS validate thecollections.
In addition to the collection in the field, ONS also undertakes what’s known as ‘central’ collection.
This is conducted ‘in house’ by ONS staff within the CPI production branch and uses various modes,
including email and telephone, online collection, and brochures.
b) European Comparison Programme/International Comparison Program
Participating countries collect prices for a basket of goods and services over a three-year rolling
program (ICP recently aligned to ECP) with two surveys being conducted in any given year. In theintervening years of t+1 and t+2, movements in the CPI are used to extrapolate forward. The
consumer surveys are conducted in the capital city, in the case of the UK, London.

4.

Statistical operations

4.1 Data collection
Integration of the data collection processes for the CPI and ECP/ICP should be relatively straightforward for those items whose price data could be used for both outputs. Historically, collection
for both outputs of the CPI and ECP/ICP use the same device in their price collection. The same
external company that collects data for the CPI supports the devices for both outputs.
Work is currently underway within the UK PPP team to utilize the CPI data to its full capacity. For
the CPI, 180,000 prices are collected monthly on a specific day and supplied in two batches. The
second is a “mop up” exercise. Around 750 items are priced, providing more than two million pricequotes. Currently around 30,000 price quotes are collected for the six ECP Consumer Goods andServices surveys (over a three-year rolling program) for the UK.
There is the potential to use this significant and timely dataset for ECP/ICP purposes. For the current
PPP consumer survey of ‘House and Garden’ matching is being undertaken; the aim going forward
is to identify those items where synergy exists and, once identified, CPI data will be used alongside
ECP/ICP price data. Initially this ‘matching exercise’ will be time consuming as primarily it is amanual task but once completed it can be used as a template going forward as only new item
proposals for both the CPI and ECP/ICP will need to be matched.
The determination of a successful matching of an item will be dependent on several criteria,
including the relative homogeneity of the price relative to others in the same basic heading, thevariation coefficient, use of proxy items etc.
In the medium to longer term however, it is envisaged that scanner data, particularly from the key
53

supermarkets in the UK market, will be used for ECP/ICP purposes and work is progressing quickly
on this significant development in price statistics. These will be millions of transaction prices andwill be used for both CPI and ECP/ICP outputs. New systems for these are already in the designphase, with an infrastructure in place to allow for the ingestion of data from the key retailers.
4.2 Data storage and editing procedures
Currently there is no integration of CPI and ECP/ICP data storage and edit processes. The ECP item
list compilation, data entry, validation and aggregation are designed, owned, and supported byEurostat. The software is internet based and is accessed through a secure system. There are threemain tools, namely the Item List Management Tool (ILMT), Data Entry Tool (DET) and Validation
Tool (VT). The DET is a standalone application which must be installed on an individual’s PC. The
only requirement is that JAVA is needed. There is also the Metadata Editing Tool (MET) which allows
for the uploading of survey reports. Access to CIRCABC is also required (CIRCABC provides a secured
working area to share information of users and interest groups). All countries participating in the
ECP submit and edit their data and metadata in this way.
This is in complete contrast to the CPI which uses internally developed and supported systems. Data
are transmitted from a third-party company via a secure SharePoint (FTP) site hosted in the UK. The
SharePoint service is provided by the third-party company. Data are validated before the point ofentry into the ONS and then again once the data has been received from the third party.

5.

Administrative structures

5.1 Funding arrangements
The UK is primarily currently funded by Eurostat to deliver the outputs of the ECP. The funding
amounts to 70% of the total cost of delivering all the outputs. However, there is a risk that if the
UK leaves with a ‘no deal’ Brexit, funding will be withdrawn. It is therefore imperative that going
forward that the UK utilizes its current resources, including price data to its full effect.
5.2 Establishment of ECP/ICP units within CPI structure
Because of the close synergies between ECP/ICP and the CPI, there are moves underway tointegrate the PPP and CPI teams within ONS Prices division. This is overlap of work at a fundamental
level, primarily the price collection level, which is the most resource intensive.

6.

Benefits of CPI-ICP integration

6.1 Enhanced sustainability of International Comparison Program
The major benefit of integration of the CPI-ECP/ICP is one of resource, both financially and time,
which in turn would ensure the sustainability of the ICP. The aim would be to collect a price that
could be used for multiple purposes.

546.2 Improvements between ICP and CPI processes
From a UK perspective at least, improvements from the CPI will flow into the ECP/ICP rather thanvice versa. Historically, the CPI has always taken priority regarding finances, time, anddevelopments. As such, and to fully utilize the significant amount of price data available, the ONS
is actively developing the way prices are collected using alternative data sources as opposed to thecurrent method of collecting prices for both CPI and ECP/ICP via the high street ‘on foot’, online(via company websites) or telephoning retailers and service providers.
6.3 Statistical capability building
It is important that any organization ensures that it continues to develop and build its statisticalcapability. Economics Statistics within ONS, the arm of which is responsible for producing economic
statistics, including consumer price collection for the CPI and ECP, is an environment that is everchanging. Economics Statistics is currently undergoing a transformation that will define a structurethat will allow ONS to deliver its commitments whilst adhering to the constraints of the budget. The
aim is to have a committed and highly skilled workforce that will deliver what stakeholders require
and in a timely manner.
Within the remit of the European Comparison Programme, the UK has through grant funding
continued to develop its knowledge and further its capability by sharing best practice with several
countries within the Programme. The UK has both hosted and visited many countries such as Spain,
France, Denmark, The Netherlands, Germany, Portugal and Ireland. This engagement withcountries within the Programme is instrumental in building the capability of the UK PPP and indeed
other countries and is a practical solution to helping those less developed countries in terms ofpractices and processes.

7.

Conclusions and future plans

If synergy exists between the CPI and ECP/ICP then there is a greater chance of successful
integration, even if that integration is limited. This is important to ensure the sustainability andfuture of the ECP/ICP.
It is also important to note that from a UK perspective integration is more likely to come from theuse of alternative data sources such as web-scraped and scanner data as opposed to the more
traditional sources (as already covered above). To enable this to happen the following needs to beactioned:•

Look at further integrating the CPI and ECP/ ICP, outside the spatial adjustment factor
exercise which is conducted once every six years.

•

Continue to logistically situate the teams responsible for both PPPs and CPI within the samecommand. This close relationship is key in ensuring the success of any future integration of
PPPs and CPI for the purpose of price collection. The intention is to amalgamate the CPI
and PPP teams under the command of one statistician, whereas currently it is under thecommand of two separate ones. These close links will bring price collection for consumer
55

goods and services under one ‘umbrella’ which by default should bring synergies between
CPI and PPPs.
•

Identify those key retailers and service providers that appear to have national pricing.
National pricing is where a retailer or service provider regardless of where it is locatedregionally will charge the same price for a good or service. Conduct analysis on the priceobservations. This could benefit both CPI and PPPs.

•

In addition to integrating CPI and ECP/ICP is it imperative that any new developments intechnology is considered when collecting prices with the aim of using the same prices fordifferent outputs. ‘The aim is for one price to fit all’.

•With improved synergy or integration between ECP/ICP and CPIs will come the significant
benefit of being able to produce regional price levels which will be able to demonstrate thedifference in price levels, relatively speaking, between regions and, in the case of the UK,composite countries of England, Wales, Northern Ireland, and Scotland.

•

By using the three-year rolling approach, Eurostat can make annual comparisons thatprovide PPPs and real expenditures for each level of aggregation, including basic heading,up to the GDP level. However, in the interim years of a survey which is conducted in periodt, the intervening t+1 and t+2 data are extrapolated using the movements in the CPI data.
Fortunately, both PPPs and CPIs are classified according to Classification Of Individual
Consumption by Purpose (COICOP) which makes this exercise relatively straightforward.

56

CPI/ICP case study: South Africa
1. Introduction
In South Africa, the statistics office used its own staff, processing systems and resources to collect
ICP pricing data. In doing so, it identified elements of the ICP process that have been implemented
as improvements for its own CPI. These improvements will further streamline collection of ICP pricedata in future rounds.

2. Background
The methodology employed in collecting data should always be tailored to the purpose of theintended output. While the inputs to a CPI and PPP are ostensibly the same – prices – their intended
use sets up differentiated requirements for their collections. For this reason, it is useful to compare
the definitions and purposes of a CPI and PPP.
In most African countries the ICP was funded by the World Bank and the African Development Bank(AfDB) as a separate program, enabling extensive country participation. However, this may have
led to the ICP being seen as an add-on with limited incentive to integrate it into existing operations.
South Africa, on the other hand, used its own permanent staff for ICP operations and did not
request or receive any AfDB funding. Improved integration of the ICP work with the regular CPI
work was therefore necessary to ensure achievement of both areas of work.

3. Product selection
The South African ICP basket was determined by first checking the common products in the CPI and
ICP baskets. The data are extracted from the CPI database and specific restrictions (e.g. outlet type,units of measurement, brand, etc.) are filtered per product. This process ensures that the exact
required ICP product is matched with the correct CPI product.
The other products might be available in the country but not in the CPI basket. These products arecollected by price collectors during the CPI collection as an extra collection. It was possible toaccommodate this additional work into the schedules of the permanent CPI price collectors.
During the ICP 2011 round, there were 1,032 household products to be collected. South Africa
collected a total of 585 products, of which 369 were common ICP-CPI products and the remaining
216 were collected in addition.
As a result of the different approach to specification, it happens that a number of ICP products may
be priced under a single CPI product description. For example, Basmati, white and Jasmine rice are
separate ICP products but are all eligible for the CPI category of rice. Consequently, a manual
matching process is undertaken in the first month of the ICP round to identify relevant items priced
for the CPI.
Due to the greater flexibility in product choice for the CPI, product specifications such as brand andproduct names are captured in a free text description field. This gives rise to numerous spelling
errors and reduces our ability to conduct deeper analysis of price behavior of identical products.
However, the stricter ICP approach has illustrated the benefits of ensuring standardized capturing
57

of names and product sizes. Revisions are currently being made to the CPI forms and capturingsystem to provide for drop down lists to provide for this. It is anticipated that this improvement willassist ICP validation in future rounds.

4. Outlet selection
Together with the price and the product specifications, the outlet type is a key characteristic for ICP
collection (at least in Africa). To maintain comparability across countries, prices should be collectedfrom the same outlet type.
For CPI, type of outlet is not specified, and purposive sampling is used for selection of outlets. Thatis, any outlet that carries the required product may be selected.
This restriction on the type of outlet in ICP limits the number of products that can be used from the
CPI. Even if the products are available in the CPI basket or in the country, they might not be includedin the country’s ICP basket because of a mismatch of type of outlet.
It was for this reason that South Africa decided to include a field for type of outlet in the household
expenditure survey and the CPI collection forms following the 2012 reweighting exercise.
Categories of chain store, independent stores and informal markets were introduced. Despite thefact that these outlet types are not identical to the ICP (supermarket, specialized stores,
department stores, wholesale stores, discount stores etc.), they do provide a step towards betteralignment and allow for analysis of pricing behavior across different outlet types. There is still
opportunity for expanding the outlet types for CPI collection since this will ensure that the prices
are not biased to specific types of outlets. We have noted that most prices in the CPI are from chain
outlets since these are pervasive and easy to collect from.

5. Price collection
Prices are collected by both field-based agents (mainly for goods) as well as head office staff (mainly
services).
The collection of data in the CPI utilizes the following forms:•

Outlet Cover Page, used to capture data on the outlet status, address, the responsible
person for collection and quality control.

•

Structured product description form (SPD), used to initiate a new product selected for
pricing according to specifications.

•Pricing form, used to price products that were previously initiated (selected for pricing).

These forms were designed during a substantial re-engineering of the CPI in 2005. The SPD form
was adopted from the ICP 2005 round and it has been extremely valuable for product initiation. A
variation of this is now also used in the producer price index. Even though the ICP uses spreadsheets
for collection, the fields in the spreadsheet correspond to the survey forms designed for the CPI.
For the ICP products which do not form part of the CPI, SPD forms are created and sent to the teamsin the field to collect from the same outlets where they are collecting CPI prices. If these products
58were not available in the existing CPI outlets, new outlets were used. The same CPI publicity
methodology is used by fieldworkers when approaching those outlets.
Integration between CPI and ICP is cost effective since there is no need to employ extra collectors
and build new systems. However, it is important that price collectors are trained thoroughly for ICP
collection so that they understand the different product specifications. Poor understanding on thepart of fieldworkers causes delays in the field and results in uncertainties, leading to the wrong item
being priced or no price being collected at all.

6. Data processing
After collection, both CPI and ICP forms from the different regions are sent to head office to be
captured. Forms are captured on the CPI database, where all editing processes are done following
the CPI methodology. At the end of editing, a clean database is ready for any type of analysis.
Although the ICP and CPI use different product codes, a correspondence table allows specific CPI
items to be extracted using a custom-built SAS program. ICP designed a data-entry sheet to be used
for capturing all collected prices. It aims to ensure a homogeneous structure of the datasets acrossall countries. Following capturing, data are merged by an additional tool which gives as output acountry data file.
This data entry sheet is not used in South Africa for capturing purposes to avoid creating different
databases for CPI and ICP. Rather, everything is captured in the CPI database and entries required
for the ICP are extracted to the ICP data-entry sheet. This means that only data that have passed
the CPI quality-validation processes are imported into the ICP data-entry sheet. This process resultsin speedier validation of ICP data.
The data entry sheet provided lessons for the CPI. Since the sheet contains protected cells to avoid
mistakes and guides the user during the entry of data, these functions have now been built into thereviewed CPI capturing system where many of the fields will now have drop down lists to choose
from. In this way, errors are avoided at an early stage.

7. Conclusion
Resistance to change and integration is common because introducing something new is usuallyperceived as extra work. Integration between the CPI and ICP in South Africa has proven to be avaluable exercise. Integration has allowed the ICP collection to be conducted with a minimum ofadditional effort and no additional resources. It has demonstrated areas for improvement of CPI
systems which enhance the quality and range of pricing data. ICP has further provided opportunities
for research work such as calculating sub-national PPPs.

59