Source document: https://docs-lawep.s3.us-east-2.amazonaws.com/1698830958287.pdf (Saint Kitts and Nevis)
e c i t c a r p d n a y r o e h T : : T h e o r y a n d p r a c t i c e The consumer price index (CPI) measures the rate at which the prices of consumer goods and services are changing over time.
It is a key statistic for purposes of economic and social policy-making, especially monetary policy and social policy, and has substantial and wide-ranging implications for governments, businesses and workers, as well as households.

This important and comprehensive manual provides guidelines for statistical offices and other agencies responsible for constructing CPIs and explains in depth the methods that are used to calculate a CPI.
It also examines the underlying economic and statistical concepts and principles needed for making choices in efficient and cost-effective ways and for appreciating the full implications of those choices.
The following international organizations, concerned both with the measurement of inflation and with policies designed to control it, have collaborated on the preparation of this manual: the International Labour Office; the International Monetary Fund; the Organisation for Economic and Co-operation and Development; the Statistical Office of the European Communities (Eurostat); the United Nations Economic Commission for Europe; and the World Bank.
3 165140 163361 ISBN 92-2-113699-X Price: 200 francs Consumer price index manual T h e o r y a n d p r a c t i c e International Labour Office International Monetary Fund Organisation for Economic Co-operation and Development Statistical Office of the European Communities (Eurostat) United Nations The World Bank Copyright # 2004 International Labour Organization/International Monetary Fund/Organisation for Economic Co-operation and Development/Statistical Oﬃce of the European Communities/United Nations/The International Bank for Reconstruction and Development/The World Bank First published 2004 Publications of the International Labour Oﬃce, the International Monetary Fund, the Organisation for Economic Co-operation and Development, the Statistical Oﬃce of the European Communities (Eurostat), the United Nations Economic Commission for Europe and the World Bank (the publishing organizations) enjoy copyright under Protocol 2 of the Universal Copyright Convention.
Nevertheless, short excerpts from them may be reproduced without authorization, on condition that the source is indicated.

For rights of reproduction of this English original, or of translation into languages other than French and Spanish, application should be made to the Publications Bureau (Rights and Permissions), International Labour Oﬃce, CH-1211 Geneva 22, Switzerland.

The International Labour Oﬃce welcomes such applications.
For rights of reproduction of the French and Spanish translations, application should be made to the International Monetary Fund, 700 19th Street, N.W.

Washington, DC, 20431, United States.
Libraries, institutions and other users registered in the United Kingdom with the Copyright Licensing Agency, 90 Tottenham Court Road, London W1P 4LP [Fax: (+44) (0) 207 631 5500; email: cla@cla.co.uk], in the United States with the Copyright Clearance Center, 222 Rosewood Drive, Danvers, MA 01923 [Fax (+1) (978) 750 4470; email: info@copyright.com], or in the other countries with associated Reproduction Rights Organizations, may make photocopies in accordance with the licences issued to them for this purpose.
ILO/IMF/OECD/UNECE/Eurostat/The World Bank Consumer price index manual: Theory and practice Geneva, International Labour Oﬃce, 2004 Guide, consumer price index, data collecting, statistical method, calculation, methodology, developed country, developing country.
09.02 ISBN 92-2-113699-X ILO Cataloguing in Publication Data The designations employed in this publication, which are in conformity with the practice of the publishing organizations, and the presentation of material therein do not imply the expression of any opinion whatsoever on the part of the publishing organizations concerning the legal status of any country, area or territory or of its authorities, or concerning the delimitation of its frontiers.
The responsibility for opinions expressed in signed articles, studies and other contributions rests solely with their authors, and publication does not constitute an endorsement by the publishing organizations of the opinions expressed in them.

Reference to names of ﬁrms and commercial products and processes does not imply their endorsement by the publishing organizations, and any failure to mention a particular ﬁrm, commercial product or process is not a sign of disapproval.
ILO publications can be obtained through major booksellers or ILO local oﬃces in many countries, or direct from ILO Publications, International Labour Oﬃce, CH-1211 Geneva 22, Switzerland.

Catalogues or lists of new publications are available free of charge from the above address or by email: pubvente@ilo.org Visit our website: www.ilo.org/publns Printed in Hong Kong iv FOREWORD This volume is an expanded revision of Consumer price indices: An ILO manual, published in 1989.
Through the mechanism of the Intersecretariat Working Group on Price Statistics (IWGPS), the revision has been undertaken under the joint responsibility of six international organizations: the International Labour Oﬃce (ILO); the International Monetary Fund (IMF); the Organisation for Economic Co-operation and Development (OECD); the Statistical Oﬃce of the European Communities (Eurostat); the United Nations Economic Commission for Europe (UNECE); and the World Bank.
It is also being published jointly by these organizations.

The manual contains detailed comprehensive information and explanations on compiling a consumer price index (CPI).

It provides an overview of the conceptual and theoretical issues that statistical oﬃces should consider when making decisions on how to deal with the various problems in the compilation of a CPI, and is intended for use by both developed and developing countries.
The chapters cover many topics; they elaborate on the diﬀerent practices currently in use, propose alternatives whenever possible, and discuss the advantages and disadvantages of each alternative.

Given the comprehensive nature of the manual, we expect it to satisfy the needs of many users.

The main purpose of the manual is to assist producers of consumer price indices, particularly in countries that are revising or setting up their CPIs.
It draws on a wide range of experience and expertise in an attempt to describe practical and suitable measurement methods.

It should also help countries to produce their CPIs in a more comparable way so that statistical oﬃces and international organizations can make meaningful international comparisons.

Bringing together a large body of knowledge on the subject, the manual may be used for self-learning, or as a teaching tool for training courses on the CPI.
Other CPI users, such as employers, workers, policy-makers and researchers, are also targeted.

The manual will inform them not only about the diﬀerent methods that are employed in collecting data and compiling such indices, but also of the limitations, so that the results may be interpreted correctly.
The drafting and revision have entailed many meetings over a ﬁve-year period, in which CPI experts from national statistical oﬃces, international and regional organizations, universities and research institutes have participated.

The new manual owes much to their collective advice and wisdom.

The electronic version of the manual is available on the Internet at www.ilo.org/stat.

The IWGPS views the manual as a ‘‘living document’’ that it will amend and update to address particular points in more detail.
This is especially true for emerging discussions and recommendations made by international groups reviewing the CPI, such as the International Conference of Labour Statisticians (ICLS), meetings of the International Working Group on Price Indices (the ‘‘Ottawa Group’’), and the Joint UNECE/ILO Meetings on Consumer Price Indices.

Comments on the manual are welcomed by the IWGPS, and should be sent to the ILO Bureau of Statistics (e-mail: stat@ilo.org).
They will be taken into account in any future revisions.
International Labour Oﬃce (ILO): A. Sylvester Young, Director, Bureau of Statistics International Monetary Fund (IMF): Horst Koehler, Managing Director Organisation for Economic Co-operation and Development (OECD): Enrico Giovanini, Director, Statistical Directorate Statistical Oﬃce of the European Communities (Eurostat): Bart Meganck, Director, Economic Statistics, and Economic and Monetary Convergence United Nations Economic Commission for Europe (UNECE): Heinrich Br€uungger, Director, Statistics Division World Bank: Shaida Badiee, Director, Development Data Group v CONTENTS Foreword ............................................................................................................................................................
Preface ............................................................................................................................................................... Acknowledgements ............................................................................................................................................. Reader’s guide .................................................................................................................................................... v xix xxv xxix 1 An introduction to consumer price index methodology ....................................................................................
The origins and uses of consumer price indices ............................................................................................
Choice of index number ................................................................................................................................ Price indices based on baskets of goods and services ................................................................................... Lowe indices .......................................................................................................................................... Laspeyres and Paasche indices .............................................................................................................. Decomposing current value changes using Laspeyres and Paasche indices ..........................................
Ratios of Lowe and Laspeyres indices ..................................................................................................
Updated Lowe indices ........................................................................................................................... Interrelationships between ﬁxed basket indices ..................................................................................... Young index ........................................................................................................................................... Geometric Young, Laspeyres and Paasche indices ................................................................................ Symmetric indices ..................................................................................................................................
Fixed base versus chain indices ............................................................................................................. Axiomatic and stochastic approaches to index numbers .............................................................................. First axiomatic approach .......................................................................................................................
Ranking of indices using the ﬁrst axiomatic approach .........................................................................

Some further tests ..................................................................................................................................

The stochastic approach and a second axiomatic approach .................................................................
The unweighted stochastic approach .....................................................................................................

The weighted stochastic approach ......................................................................................................... A second axiomatic approach ...............................................................................................................
Cost of living index ....................................................................................................................................... Upper and lower bounds on a cost of living index ...............................................................................
Some special cases ................................................................................................................................. Estimating COLIs by superlative indices ............................................................................................... Representativity bias .............................................................................................................................. Data requirements and calculation issues ..............................................................................................
Allowing for substitution ....................................................................................................................... Aggregation issues ......................................................................................................................................... Illustrative numerical data ............................................................................................................................
Seasonal products ..........................................................................................................................................

Elementary price indices ...............................................................................................................................
Weights within elementary aggregates ................................................................................................... Interrelationships between diﬀerent elementary index formulae ........................................................... Axiomatic approach to elementary indices ............................................................................................ Economic approach to elementary indices ............................................................................................ Concepts, scope and classiﬁcations ............................................................................................................... Acquisitions and uses .............................................................................................................................
Unconditional and conditional cost of living indices ............................................................................ Speciﬁc types of transactions ................................................................................................................. Household production ........................................................................................................................... Coverage of households and outlets ...................................................................................................... Price variation ........................................................................................................................................ Classiﬁcations ......................................................................................................................................... Consumer price indices and national accounts price deﬂators ..............................................................
Expenditure weights ...................................................................................................................................... Household expenditure surveys and national accounts ......................................................................... Other sources for estimating expenditure weights ................................................................................. Collection of price data ................................................................................................................................. Random sampling and purposive sampling .......................................................................................... vii CONTENTS Methods of price collection ................................................................................................................... Continuity of price collection ................................................................................................................ Resampling ............................................................................................................................................
Adjusting prices for quality changes .............................................................................................................

Evaluation of the eﬀect of quality change on price ..............................................................................

Implicit methods for adjusting for quality changes ...............................................................................
Explicit quality adjustments ...................................................................................................................

Item substitution and new goods .................................................................................................................. New goods and services .........................................................................................................................
Calculation of consumer price indices in practice ........................................................................................
Elementary price indices ........................................................................................................................ Higher-level indices ................................................................................................................................ Organization and management .....................................................................................................................
Publication and dissemination ...................................................................................................................... 2 Uses of consumer price indices ........................................................................................................................ A range of possible consumer price indices ..................................................................................................
Indexation .....................................................................................................................................................

Indexation of wages ...............................................................................................................................

Indexation of social security beneﬁts .....................................................................................................
The type of index used for indexation ...................................................................................................

Indexation of interest, rents and other contractual payments ...............................................................
Taxation ................................................................................................................................................. Real consumption and real income ...............................................................................................................
Consistency between price indices and expenditure series ..................................................................... Purchasing power parities ...................................................................................................................... Use of the consumer price index for accounting under inﬂation ................................................................. Current purchasing power accounts ...................................................................................................... Current cost accounting ......................................................................................................................... Consumer price indices and general inﬂation ............................................................................................... Consumer price indices and inﬂation targets ........................................................................................ Consumer price indices and international comparisons of inﬂation .....................................................
Popularity of consumer price indices as economic statistics .........................................................................
The need for independence and integrity of consumer price indices ............................................................ 3 Concepts and scope ......................................................................................................................................... Introduction ..................................................................................................................................................
Alternative consumption aggregates ............................................................................................................. Acquisitions and expenditures ............................................................................................................... Monetary versus non-monetary expenditures ........................................................................................ Acquisitions and uses ....................................................................................................................................
Durables and non-durables .................................................................................................................... Consumer price indices based on acquisitions and uses ........................................................................
Basket indices and cost of living indices ....................................................................................................... Lowe indices ..........................................................................................................................................

Cost of living indices .............................................................................................................................
Expenditures and other payments outside the scope of consumer price indices ..........................................
Transfers ................................................................................................................................................ Insurance ................................................................................................................................................ Gambling ...............................................................................................................................................
Transactions in ﬁnancial assets ..............................................................................................................
Purchases and sales of foreign currency ....................................................................................................... Payments, ﬁnancing and credit ..................................................................................................................... Financial transactions and borrowing ...................................................................................................
The creation of a ﬁnancial asset/liability ...............................................................................................
Hire purchase ......................................................................................................................................... Interest payments ................................................................................................................................... Household production .................................................................................................................................. Business activities ...................................................................................................................................
Consumption of own produce ............................................................................................................... Coverage of households and outlets ............................................................................................................. Deﬁnition of household ......................................................................................................................... Types of household ................................................................................................................................ viii CONTENTS Geographical coverage ...........................................................................................................................
Outlet coverage ...................................................................................................................................... Price variation ............................................................................................................................................... Price discrimination ............................................................................................................................... Price variation between outlets ..............................................................................................................
Outlet rotation .......................................................................................................................................
Treatment of some speciﬁc household expenditures ..................................................................................... Fees of agents and brokers .................................................................................................................... Undesirable or illegal goods and services ..............................................................................................
Luxury goods and services .................................................................................................................... Second-hand goods ................................................................................................................................ Imputed expenditures on goods and services ........................................................................................ Price coverage ................................................................................................................................................ Taxes and subsidies ................................................................................................................................
Discounts, rebates, loyalty schemes and ‘‘free’’ products ..................................................................... Classiﬁcation .................................................................................................................................................
Criteria for classifying consumption expenditure .................................................................................. Classiﬁcation by product type ............................................................................................................... Classiﬁcation by purpose ....................................................................................................................... Classiﬁcations for consumer price indices .............................................................................................
Publication level ..................................................................................................................................... Classiﬁcation of Individual Consumption according to Purpose (COICOP) ........................................
Appendix 3.1 Consumer price indices and national accounts price deﬂators ........................................ 4 Expenditure weights and their sources ............................................................................................................. Introduction ..................................................................................................................................................
The weighting structure of the consumer price index ................................................................................... Group, class and sub-class weights ........................................................................................................

Regional weights ....................................................................................................................................
Outlet or outlet-type weights .................................................................................................................
Elementary aggregate weights ................................................................................................................ Data sources .................................................................................................................................................. Household expenditure surveys ............................................................................................................. National accounts .................................................................................................................................. Retail sales data ..................................................................................................................................... Point-of-purchase surveys ...................................................................................................................... Scanner data .......................................................................................................................................... Population censuses ............................................................................................................................... Deriving the weights in practice .................................................................................................................... Payments that are not consumption expenditures ................................................................................. Unimportant expenditures ..................................................................................................................... Products that are diﬃcult to price ......................................................................................................... Use and combination of diﬀerent sources .............................................................................................
Adjusting the weights derived from household expenditure surveys .....................................................

Weight reference period .........................................................................................................................

Need for revising the weights ................................................................................................................
Frequency of updating the weights ....................................................................................................... Classiﬁcation ..........................................................................................................................................

Items requiring special treatment ..........................................................................................................
Errors in weighting ................................................................................................................................ 5 Sampling ......................................................................................................................................................... Introduction .................................................................................................................................................. Probability sampling techniques ...................................................................................................................
Implementing probability sampling in consumer price indices ..................................................................... Sampling techniques based on probability proportional to size ........................................................... Sampling methods used by the US Bureau of Labor Statistics ............................................................ Non-probability sampling techniques ........................................................................................................... Reasons for using non-probability sampling ......................................................................................... Cut-oﬀ sampling ....................................................................................................................................
Quota sampling ......................................................................................................................................
The representative item method ............................................................................................................ Sampling in time .................................................................................................................................... ix CONTENTS Choice of sampling method .......................................................................................................................... Estimation procedures ...................................................................................................................................
Implementing estimation procedures for consumer price indices ................................................................. Variance estimation ....................................................................................................................................... Variances of elementary index formulae ...............................................................................................
The United States approach ..................................................................................................................

The Swedish approach ...........................................................................................................................

The French approach ............................................................................................................................
The Luxembourg approach ................................................................................................................... Other approaches ................................................................................................................................... Optimal allocation ......................................................................................................................................... Summary ....................................................................................................................................................... 6 Price collection ................................................................................................................................................ Introduction ..................................................................................................................................................
Frequency and timing of collection ..............................................................................................................

Taking account of hyperinﬂation ..........................................................................................................
Item speciﬁcation .......................................................................................................................................... Collection procedures .................................................................................................................................... Price collection techniques ..................................................................................................................... Questionnaire design .............................................................................................................................. Field procedures ..................................................................................................................................... Central and head oﬃce collection ......................................................................................................... Price reductions ...................................................................................................................................... Price bargaining .....................................................................................................................................
Forced replacements, product substitution and quality adjustment ..................................................... Related issues ................................................................................................................................................ Electronic reporting ............................................................................................................................... Purchasing power parities ...................................................................................................................... Data quality and quality assurance .......................................................................................................
Documentation ......................................................................................................................................
Appendix 6.1 Extract from a simple price collection form ........................................................................... 7 Adjusting for quality change ........................................................................................................................... Introduction .................................................................................................................................................. ................................................................................................. Why the matched models method may fail Missing items ......................................................................................................................................... Sampling concerns ................................................................................................................................. New products .........................................................................................................................................
The nature of quality change ........................................................................................................................ A utility-based approach .......................................................................................................................
Conditional indices ................................................................................................................................ An overview of methods of quality adjustment when matched items are unavailable ..................................
Additive versus multiplicative adjustment ............................................................................................. Base versus current period adjustment ..................................................................................................

Long-run versus short-run comparisons ................................................................................................
Implicit methods of quality adjustment ........................................................................................................

Overlap ...................................................................................................................................................
Overall mean or targeted mean imputation .......................................................................................... Class mean imputation ..........................................................................................................................

Comparable replacement .......................................................................................................................
Linked to show no price change ............................................................................................................

Carry-forward ........................................................................................................................................

Explicit methods of quality adjustment ........................................................................................................
Expert judgement ...................................................................................................................................
Quantity adjustment .............................................................................................................................. Diﬀerences in production or option costs ............................................................................................. Hedonic approach ..................................................................................................................................
Limitations of the hedonic approach ....................................................................................................

Choice between quality adjustment methods ................................................................................................

High-technology and other sectors with a rapid turnover of models ...........................................................
Some examples ....................................................................................................................................... Hedonic price indices ............................................................................................................................. x CONTENTS The diﬀerence between hedonic indices and matched indices ............................................................... Chaining .................................................................................................................................................
Long-run and short-run comparisons ........................................................................................................... Quality adjustment methods in short-run comparisons ........................................................................
Implicit short-run comparisons using imputations ................................................................................ Single-stage and two-stage indices .........................................................................................................
Appendix 7.1 Data on personal computers, obtained from United Kingdom Compaq and Dell web sites, July 2000, to illustrate hedonic regression ................................................................................................. 8 Item substitution, sample space and new products ........................................................................................... Introduction ..................................................................................................................................................
Matched samples ...........................................................................................................................................

Sample space and item replacement or substitution .....................................................................................
Sample rotation, chaining and hedonic indices ............................................................................................ Information requirements for a quality adjustment strategy ........................................................................ Statistical metadata system .................................................................................................................... New products and how they diﬀer from products with quality changes ..................................................... Incorporation of new products .....................................................................................................................
Sample rebasing and rotation ................................................................................................................

Directed replacements and sample augmentation .................................................................................
Reservation prices .................................................................................................................................. Summary .......................................................................................................................................................

Appendix 8.1 Appearance or disappearance of products or outlets .............................................................
Appendix 8.2 New goods and substitution ................................................................................................... 9 Calculating consumer price indices in practice ................................................................................................ Introduction ..................................................................................................................................................
The calculation of price indices for elementary aggregates .......................................................................... Construction of elementary aggregates .................................................................................................. Construction of elementary price indices .............................................................................................. Chain versus direct indices for elementary aggregates ..........................................................................
Consistency in aggregation ....................................................................................................................
Missing price observations ..................................................................................................................... Other formulae for elementary price indices ......................................................................................... Unit value indices .................................................................................................................................. Formulae applicable to scanner data ....................................................................................................
The calculation of higher-level indices ........................................................................................................... Consumer price indices as weighted averages of elementary indices ............................................................ A numerical example ............................................................................................................................. Young and Lowe indices ....................................................................................................................... Factoring the Young index .................................................................................................................... Price-updating from the weight reference period to the price reference period ....................................
The introduction of new weights and chain linking .............................................................................. Decomposition of index changes ...........................................................................................................
Some alternatives to ﬁxed weight indices .............................................................................................. Data editing ................................................................................................................................................... Identifying possible errors and outliers .................................................................................................
Verifying and correcting data ................................................................................................................ 10 Some special cases ........................................................................................................................................ Introduction .................................................................................................................................................. Owner-occupied housing ............................................................................................................................... Use ......................................................................................................................................................... Payments ................................................................................................................................................ Acquisitions ...........................................................................................................................................
Clothing .........................................................................................................................................................

The clothing market ...............................................................................................................................

Approaches to constructing indices for non-seasonal clothing ..............................................................
Replacement of items and quality change .............................................................................................
Approaches to including seasonal clothing in the consumer price index .............................................. Summary comments ............................................................................................................................... Telecommunication services .......................................................................................................................... Representative items – matched samples ............................................................................................... Representative items – unit values ......................................................................................................... xi CONTENTS Customer proﬁles ...................................................................................................................................
Sample of bills ....................................................................................................................................... Financial services .......................................................................................................................................... Currency exchange ................................................................................................................................. Stockbroking services ............................................................................................................................. Deposit and loan facilities ..................................................................................................................... Real estate agency services ............................................................................................................................ Property insurance services ........................................................................................................................... Payments ................................................................................................................................................ Use ......................................................................................................................................................... Acquisitions ........................................................................................................................................... Pricing gross insurance premiums .........................................................................................................
Using gross premiums as a proxy for the net insurance service ...........................................................
Appendix 10.1 Calculation of a price index for a deposit product .............................................................. 11 Errors and bias .............................................................................................................................................. Introduction .................................................................................................................................................. Types of error ............................................................................................................................................... Sampling error ....................................................................................................................................... Non-sampling error ...............................................................................................................................
Measuring error and bias .............................................................................................................................. Estimation of variance ........................................................................................................................... Qualitative descriptions of non-sampling errors ....................................................................................
Procedures to minimize errors ...................................................................................................................... Types of bias .................................................................................................................................................
Components of bias ...................................................................................................................................... Upper-level substitution bias .................................................................................................................
Elementary aggregate bias ..................................................................................................................... Quality change and new products bias .................................................................................................. New outlet bias ...................................................................................................................................... Summary of bias estimates ............................................................................................................................
Conclusion ..................................................................................................................................................... 12 Organization and management ...................................................................................................................... Introduction .................................................................................................................................................. Local collection .............................................................................................................................................
Contracting out ...................................................................................................................................... Central collection ................................................................................................................................... Quality in the ﬁeld ........................................................................................................................................ Descriptions ........................................................................................................................................... Continuity .............................................................................................................................................. Data entry queries .................................................................................................................................
Feedback ................................................................................................................................................ Quality checks in local collection: The role of auditors ...............................................................................
Monitoring ............................................................................................................................................. Backchecking ......................................................................................................................................... Other auditor functions ......................................................................................................................... Quality checks in head oﬃce ......................................................................................................................... Reports ................................................................................................................................................... Algorithms .............................................................................................................................................
Producing and publishing the index .............................................................................................................

Monthly compilation ............................................................................................................................. Spreadsheets ...........................................................................................................................................
Introducing changes ...............................................................................................................................
Disaster recovery ................................................................................................................................... Quality management and quality management systems ............................................................................... Quality management systems .................................................................................................................
Scope for greater use of quality management techniques .....................................................................
Performance management, development and training .................................................................................. Training requirements ............................................................................................................................ Speciﬁc training for compilers and collectors ........................................................................................
Documentation ......................................................................................................................................
Reviews .................................................................................................................................................. xii CONTENTS 13 Publication, dissemination and user relations ................................................................................................ Introduction .................................................................................................................................................. Time series presentation of level and change ................................................................................................
Seasonal adjustment and smoothing of the index ........................................................................................

Analysis of contributions to change ............................................................................................................. Economic commentary and interpretation of the index ...............................................................................
Presentation of related or alternative measures ............................................................................................

Core inﬂation .........................................................................................................................................
Alternative indices ................................................................................................................................. Sub-aggregate indices ............................................................................................................................. Press release, bulletin and methodological statement ................................................................................... International standards concerning the dissemination of consumer price indices ........................................
Timing of dissemination of the consumer price index .................................................................................. Timeliness of release versus data accuracy ................................................................................................... Access to data ............................................................................................................................................... Conﬁdentiality ............................................................................................................................................... Electronic dissemination ...............................................................................................................................
User consultation .......................................................................................................................................... Diﬀerent uses of consumer price indices ...............................................................................................
Presentation of methodology ................................................................................................................. Role of advisory committees .................................................................................................................
Explaining index quality ........................................................................................................................ 14 The system of price statistics ........................................................................................................................ Introduction .................................................................................................................................................. National accounts as a framework for the system of price statistics ...........................................................
Aggregate supply and use of goods and services .................................................................................. Institutional units and establishments ................................................................................................... Accounts of institutional units ..............................................................................................................
The consumer price index among major price indices ..................................................................................

Scope of the expenditure aggregates of the consumer price index ........................................................

The consumer price index as a measure of inﬂation in market transactions ........................................
Treatment of cross-border shopping in the consumer price index ........................................................ Other price indicators in the national accounts ............................................................................................ Price indices for total supply ................................................................................................................. Price indices for intermediate consumption ........................................................................................... Price indices for ﬁnal uses ..................................................................................................................... Price indices for gross domestic product ............................................................................................... Price indices for labour services .............................................................................................................
Framework for a system of price statistics for goods and services .............................................................. International comparisons of expenditure on goods and services ................................................................ 15 Basic index number theory ............................................................................................................................ Introduction ..................................................................................................................................................
The decomposition of value aggregates into price and quantity components ..............................................

The decomposition of value aggregates and the product test ...............................................................
The Laspeyres and Paasche indices ....................................................................................................... Symmetric averages of ﬁxed basket price indices .........................................................................................

The Fisher index as an average of the Paasche and Laspeyres indices .................................................
The Walsh index and the theory of the ‘‘pure’’ price index ..................................................................

Annual weights and monthly price indices ...................................................................................................

The Lowe index with monthly prices and annual base year quantities ................................................
The Lowe index and mid-year indices ...................................................................................................

The Young index ...................................................................................................................................

The Divisia index and discrete approximations to it ....................................................................................
The Divisia price and quantity indices .................................................................................................. Discrete approximations to the continuous time Divisia index ............................................................

Fixed base versus chain indices .............................................................................................................
Appendix 15.1 The relationship between the Paasche and Laspeyres indices ..............................................

Appendix 15.2 The relationship between the Lowe and Laspeyres indices ..................................................

Appendix 15.3 The relationship between the Young index and its time antithesis ......................................
Appendix 15.4 The relationship between the Divisia and economic approaches ......................................... xiii CONTENTS 16 The axiomatic and stochastic approaches to index number theory ................................................................ Introduction ..................................................................................................................................................
The levels approach to index number theory ............................................................................................... An axiomatic approach to unilateral price indices ................................................................................ A second axiomatic approach to unilateral price indices ......................................................................
The ﬁrst axiomatic approach to bilateral price indices .................................................................................
Bilateral indices and some early tests .................................................................................................... Homogeneity tests .................................................................................................................................. Invariance and symmetry tests ..............................................................................................................
Mean value tests .................................................................................................................................... Monotonicity tests .................................................................................................................................

The Fisher ideal index and the test approach .......................................................................................
The test performance of other indices ...................................................................................................

The additivity test ..................................................................................................................................

The stochastic approach to price indices ......................................................................................................
The early unweighted stochastic approach ............................................................................................

The weighted stochastic approach .........................................................................................................

The second axiomatic approach to bilateral price indices ............................................................................
The basic framework and some preliminary tests ................................................................................. Homogeneity tests .................................................................................................................................. Invariance and symmetry tests .............................................................................................................. A mean value test .................................................................................................................................. Monotonicity tests ................................................................................................................................. Weighting tests .......................................................................................................................................
The To¨ rnqvist–Theil price index and the second test approach to bilateral indices .............................

The test properties of the Lowe and Young indices .....................................................................................
Appendix 16.1 Proof of the optimality of the To¨ rnqvist–Theil price index in the second bilateral test approach ................................................................................................................................................... 17 The economic approach to index number theory: The single-household case ................................................. Introduction ..................................................................................................................................................
The Konu¨ s cost of living index and observable bounds ...............................................................................
The true cost of living index when preferences are homothetic ................................................................... Superlative indices: The Fisher ideal index ................................................................................................... Quadratic mean of order r superlative indices .............................................................................................. Superlative indices: the To¨ rnqvist index .......................................................................................................
The approximation properties of superlative indices .................................................................................... Superlative indices and two-stage aggregation ..............................................................................................

The Lloyd–Moulton index number formula .................................................................................................
Annual preferences and monthly prices ........................................................................................................

The Lowe index as an approximation to a true cost of living index .................................................... A ﬁrst-order approximation to the bias of the Lowe index .................................................................. A second-order approximation to the substitution bias of the Lowe index .........................................
The problem of seasonal commodities ..................................................................................................
The problem of a zero price increasing to a positive price ........................................................................... 18 The economic approach to index number theory: The many-household case .................................................. Introduction .................................................................................................................................................. Plutocratic cost of living indices and observable bounds .............................................................................
The Fisher plutocratic price index ................................................................................................................ Democratic versus plutocratic cost of living indices ..................................................................................... 19 Price indices using an artiﬁcial data set ........................................................................................................ Introduction ..................................................................................................................................................
The artiﬁcial data set ....................................................................................................................................
Early price indices: The Carli, Jevons, Laspeyres and Paasche indices ........................................................ Asymmetrically weighted price indices .......................................................................................................... Symmetrically weighted indices: Superlative and other indices .................................................................... Superlative indices constructed in two stages of aggregation ....................................................................... Lloyd–Moulton price indices ........................................................................................................................
Additive percentage change decompositions for the Fisher ideal index .......................................................

The Lowe and Young indices .......................................................................................................................
Mid-year indices based on the Lowe formula .............................................................................................. Young-type indices ........................................................................................................................................ xiv CONTENTS 20 Elementary indices ........................................................................................................................................ Introduction ..................................................................................................................................................
Ideal elementary indices ................................................................................................................................ Aggregation and classiﬁcation problems for elementary aggregates ............................................................

Elementary indices used in practice ..............................................................................................................
Numerical relationships between the frequently used elementary indices ....................................................

The axiomatic approach to elementary indices .............................................................................................

The economic approach to elementary indices .............................................................................................
The sampling approach to elementary indices ..............................................................................................

The use of scanner data in constructing elementary aggregates ................................................................... A simple stochastic approach to elementary indices .....................................................................................
Conclusion ..................................................................................................................................................... 21 Quality change and hedonics ......................................................................................................................... Introduction .................................................................................................................................................. New and disappearing items ......................................................................................................................... Hedonic prices and implicit markets .............................................................................................................
Items as tied bundles of characteristics .................................................................................................

The consumer or demand side ...............................................................................................................
The producer or supply side ................................................................................................................. Equilibrium ............................................................................................................................................ What hedonic prices mean ..................................................................................................................... An alternative, consumer-based hedonic theoretical formulation ......................................................... Hedonic indices .............................................................................................................................................
Theoretical characteristics price indices ................................................................................................. Hedonic regressions and dummy variables of time ............................................................................... Hedonic imputation indices ................................................................................................................... Superlative and exact hedonic indices ................................................................................................... Unweighted hedonic indices and unweighted matched index number formulae ................................... New goods and services ................................................................................................................................
Appendix 21.1 Some econometric issues ....................................................................................................... 22 The treatment of seasonal products ............................................................................................................... Introduction .................................................................................................................................................. A seasonal commodity data set .................................................................................................................... Year-over-year monthly indices .................................................................................................................... Year-over-year annual indices ....................................................................................................................... Rolling year annual indices ...........................................................................................................................
Predicting a rolling year index using a current period year-over-year monthly index .................................

Maximum overlap month-to-month price indices ........................................................................................

Annual basket indices with carry forward of unavailable prices ..................................................................
Annual basket indices with imputation of unavailable prices ...................................................................... Bean and Stine Type C or Rothwell indices ................................................................................................. Forecasting rolling year indices using month-to-month annual basket indices ............................................
Conclusion ..................................................................................................................................................... 23 Durables and user costs ................................................................................................................................. Introduction ..................................................................................................................................................
The acquisitions approach ............................................................................................................................

The rental equivalence approach ..................................................................................................................

The user cost approach .................................................................................................................................
The relationship between user costs and acquisition costs ...........................................................................
Alternative models of depreciation ............................................................................................................... A general model of depreciation for (unchanging) consumer durables ................................................ Geometric or declining balance depreciation ........................................................................................ Straight line depreciation ....................................................................................................................... ‘‘One hoss shay’’ or light bulb depreciation ..........................................................................................
Unique durable goods and the user cost approach .....................................................................................

The user cost of owner-occupied housing .....................................................................................................

The treatment of costs that are tied to owner-occupied housing .................................................................
The treatment of mortgage interest costs ..............................................................................................

The treatment of property taxes ............................................................................................................
The treatment of property insurance ..................................................................................................... xv CONTENTS The treatment of maintenance and renovation expenditures ...............................................................
The treatment of the transactions costs of home purchase ...................................................................
User costs for landlords versus owners ......................................................................................................... Damage costs ......................................................................................................................................... Non-payment of rent and vacancy costs ...............................................................................................
Billing and maintenance costs ............................................................................................................... ............................................................................................................

The opportunity cost of capital .......................................................................
The supply of additional services for rented properties The payments approach ................................................................................................................................

Alternative approaches for pricing owner-occupied housing ........................................................................

The acquisitions approach .....................................................................................................................
The rental equivalence approach ...........................................................................................................

The user cost approach .......................................................................................................................... A glossary of main terms ...............................................................................................................................

Appendix to the glossary.
Some basic number formulae and terminology .......................................... Annex 1 Harmonized Indices of Consumer Prices (European Union) ............................................................ Annex 2 Classiﬁcation of Individual Consumption according to Purpose (COICOP)-Extract ....................... Annex 3 Resolution concerning consumer price indices adopted by the Seventeenth International Conference of Labour Statisticians, 2003 ................................................................................................................ Annex 4 Spatial comparisons of consumer prices, purchasing power parities and the International Comparison Program ........................................................................................................................................... Bibliography ................................................................................................................................................... Index ..............................................................................................................................................................
List of tables 4.1 Example of weights by region and outlet type for the sub-class ‘‘fresh fruit’’ ............................... 5.1 Systematic sample of 3 out of 10 outlets, based on probability proportional to size .................... 5.2 Pareto sample of 3 out of 10 outlets, based on probability proportional to size .......................... 6.1 Example of a survey form showing the number of price quotations by shop or stall ................... 6.2 Example illustrating the method for determining the actual price paid by the purchaser when bargaining takes place .................................................................................................................... 7.1 Example of the implicit methods of quality adjustment ................................................................ 7.2 Example of the bias from implicit quality adjustment when the (mean) price change of quality- adjusted new items compared with the items they are replacing is assumed not to change (r2=1.00) ......................................................................................................................................... 7.3 Example of size, price and unit price of bags of ﬂour ................................................................... 7.4 Hedonic regression results for Dell and Compaq personal computers .......................................... 7.5 Example of long-run and short-run comparisons ........................................................................... 8.1 Example of sample augmentation ................................................................................................... 9.1 Calculation of price indices for an elementary aggregate ............................................................... 9.2 Imputation of temporarily missing prices ....................................................................................... 9.3 Disappearing items and their replacements with no overlap .......................................................... 9.4 Disappearing and
 replacement items with overlapping prices ........................................................ 9.5 The aggregation of elementary price indices .................................................................................. 9.6 Price-updating of weights between the weight and price reference periods ................................... 9.7 Calculation of a chain index ........................................................................................................... 9.8 Decomposition of index changes .................................................................................................... 10.1 Calculation of a mortgage debt series ............................................................................................ 10.2 Calculation of a mortgage interest charges series ........................................................................... 10.3 Synthetic price data to illustrate approaches to constructing clothing price indices ...................... 10.4 Alternative price indices for summer seasonal clothing ................................................................. 10.5 Alternative price indices for winter seasonal clothing .................................................................... 10.6 Alternative price indices for total clothing ..................................................................................... 10.7 An illustrative index structure for telecommunication services (representative item approach) .... 10.8 Examples of
 speciﬁcations of telecommunication services ............................................................. 10.9 Example of a user proﬁle for mobile phone services ...................................................................... xvi CONTENTS 10.10 Illustration of the impact of taxes on measures of insurance services
 ............................................. 11.1 A taxonomy of errors in a consumer price index ........................................................................... 14.1 Production account for an establishment, institutional unit or institutional sector ....................... 14.2 Production account with product detail for an establishment or local kind of activity unit ......... 14.3 Use of income account for institutional units and sectors ............................................................. 14.4 Use of income account with product detail for institutional units and sectors ............................. 14.5 Use of income account with product detail for the total economy ................................................ 14.6 Capital account ............................................................................................................................... 14.7 Capital account with product detail ............................................................................................... 14.8 External account of goods and services .......................................................................................... 14.9 External account of goods and services with product detail .......................................................... 14.10 The supply and use table (SUT) ..................................................................................................... 14.11 Location and coverage of major price indices: Columns in the supply and use table ................... 14.12 Deﬁnition of scope, price relatives, coverage and weights for major price indices ........................ 14.13 Generation of income account for establishment, institutional unit or institutional sector .......... 14.14 Generation of income account for establishment and industry with labour services (occupational) detail
 ............................................................................................................................................... 14.15 A framework for price statistics ..................................................................................................... 19.1 Prices for six commodities .............................................................................................................. 19.2 Quantities for six commodities ....................................................................................................... 19.3 Expenditures and expenditure shares for six commodities ............................................................. 19.4 The ﬁxed base Laspeyres, Paasche, Carli and Jevons indices ........................................................ 19.5 Chain Laspeyres, Paasche, Carli and Jevons indices ...................................................................... 19.6 Asymmetrically weighted ﬁxed base indices ................................................................................... 19.7 Asymmetrically weighted indices using the chain principle ............................................................ 19.8 Asymmetrically weighted ﬁxed base indices for commodities 3–6 ................................................. 19.9 Asymmetrically weighted chained indices for commodities 3–6 ..................................................... 19.10 Symmetrically weighted ﬁxed base indices ..................................................................................... 19.11 Symmetrically weighted indices using the chain principle .............................................................. 19.12 Fixed base superlative single-stage and two-stage indices .............................................................. 19.13 Chained superlative single-stage and two-stage indices .................................................................. 19.14 Chained Fisher and ﬁxed base Lloyd–Moulton indices
 ................................................................. 19.15 Chained Fisher and chained Lloyd–Moulton indices ..................................................................... 19.16 Diewert’s additive percentage change decomposition of the Fisher index ..................................... 19.17 Van Ijzeren’s decomposition of the Fisher price index .................................................................. 19.18 The Lowe and Young indices, the ﬁxed base Laspeyres, Paasche and Fisher indices, and the chained Laspeyres, Paasche and Fisher indices .............................................................................. 19.19 The ﬁve Lowe indices, the mid-year index, and the To¨ rnqvist and Fisher chain indices .............. 19.20 The ﬁve Young-type indices and the To¨ rnqvist and Fisher chain indices ..................................... 20.1 Proportion of transactions in 2000 that could be matched to 1998 ............................................... 20.2 Laspeyres price indices by type of classiﬁcation, September 1998–September 2000 ...................... 20.3 Fisher price indices by type of classiﬁcation, September 1998–September 2000 ............................ 22.1 An artiﬁcial seasonal data set: Prices ............................................................................................. 22.2 An artiﬁcial seasonal data set: Quantities ...................................................................................... 22.3 Year-over-year monthly ﬁxed base Laspeyres indices .................................................................... 22.4 Year-over-year monthly ﬁxed base Paasche indices ....................................................................... 22.5 Year-over-year monthly ﬁxed base Fisher indices .......................................................................... 22.6 Year-over-year approximate monthly ﬁxed base Paasche indices .................................................. 22.7 Year-over-year
22.7 Year-over-year approximate monthly ﬁxed base Fisher indices ..................................................... 22.8 Year-over-year monthly chained Laspeyres indices ........................................................................ 22.9 Year-over-year monthly chained Paasche indices ........................................................................... 22.10 Year-over-year monthly chained Fisher indices ............................................................................. 22.11 Year-over-year monthly approximate chained Laspeyres indices .................................................. 22.12 Year-over-year monthly approximate chained Paasche indices ..................................................... 22.13 Year-over-year monthly approximate chained Fisher indices ........................................................ 22.14 Annual ﬁxed base Laspeyres, Paasche and Fisher price indices .................................................... 22.15 Annual approximate ﬁxed base Laspeyres, Paasche, Fisher and geometric Laspeyres indices ...... xvii CONTENTS 22.16 Annual chained Laspeyres, Paasche and Fisher price indices ........................................................ 22.17 Annual approximate chained Laspeyres, Paasche and Fisher price indices
 ................................... 22.18 Rolling year Laspeyres, Paasche and Fisher price indices ............................................................. 22.19 Rolling year approximate Laspeyres, Paasche and Fisher price indices ........................................ 22.20 Rolling year ﬁxed base Laspeyres and seasonally adjusted approximate rolling year price indices 22.21 Month-to-month maximum overlap chained Laspeyres, Paasche and Fisher price indices ........... 22.22 Month-to-month chained Laspeyres, Paasche and
 Fisher price indices ......................................... 22.23 Lowe, Young, geometric Laspeyres and centred rolling year indices with carry forward prices ... 22.24 Lowe, Young, geometric Laspeyres and centred rolling year indices with imputed prices ............ 22.25 The Lowe with carry forward prices, Rothwell and normalized Rothwell indices ........................ 22.26 Seasonally adjusted Lowe, Young and geometric Laspeyres indices with carry forward prices and the centred rolling year index ......................................................................................................... 22.27 Seasonally adjusted Lowe, Young and geometric Laspeyres indices with imputed prices, seasonally adjusted Rothwell and centred rolling year indices ........................................................................
List of ﬁgures 4.1 Typical aggregation structure of a consumer price index (CPI) ..................................................... 6.1 Price collection procedures ............................................................................................................. 7.1 Quality adjustment for diﬀerent-sized items ................................................................................... 7.2 Scatter diagram showing prices and processing speeds of personal computers ............................. 7.3 Flowchart for making decisions on quality change ........................................................................ 9.1 Typical aggregation structure of a consumer price index (CPI) ..................................................... 12.1 Price collection procedures ............................................................................................................. 17.1 The Laspeyres and Paasche bounds to the true cost of living index ............................................. 21.1 Consumption and production decisions for combinations of characteristics ................................. 22.1 Rolling year ﬁxed base and chained Laspeyres, Paasche and Fisher indices ................................. 22.2 Rolling year approximate ﬁxed base and chained Laspeyres, Paasche and Fisher indices ............ 22.3 Fixed base Laspeyres, seasonally adjusted approximate and approximate rolling year indices .... 22.4 Lowe, Young, geometric Laspeyres and centred rolling year Laspeyres indices ........................... 22.5 Lowe, Young and geometric Laspeyres with imputed prices and centred rolling year indices ..... 22.6 The Lowe and Rothwell price indices ............................................................................................ 22.7 Seasonally adjusted Lowe, Young and geometric Laspeyres indices with carry forward prices and the centred rolling year index ......................................................................................................... 22.8 Seasonally adjusted Lowe, Young and geometric Laspeyres indices with imputed prices, seasonally adjusted Rothwell and centred rolling year indices ........................................................................ A4.1 A minimum spanning tree: Europe ............................................................................................... A4.2 Price data for CPI and ICP activities ............................................................................................ A4.3 A sequence of price comparisons ...................................................................................................
List of boxes 13.1 Model presentation of consumer price index ................................................................................. 13.2 Model note on methodology – to be included in press releases on consumer price indices .......... 14.1 Institutional sectors in the System of National Accounts 1993 ....................................................... 14.2 Coverage of industries or activities by the producer price index in terms of aggregate output value 14.3 Treatment of housing and consumer durables in the system of national accounts and in consumer price indices .................................................................................................................................... xviii PREFACE The International Labour Oﬃce (ILO), the International Monetary Fund (IMF), the Organisation for Economic Co-operation and Development (OECD), the Statistical Oﬃce of the European Communities (Eurostat), the United Nations Economic Commission for Europe (UNECE) and the World Bank, together with experts from a number of national statistical oﬃces and universities, have collaborated since 1998 on developing this manual.
The sponsoring organizations endorse the principles and recommendations con- tained in it as good practice for statistical agencies in compiling their consumer price indices (CPIs).

Because of practical and resource constraints, however, some of the current recommendations may not be imme- diately attainable by all statistical oﬃces, and they should therefore serve as guidelines or targets for agencies as they revise their CPIs and improve their CPI programmes.
There are not always clear-cut solutions to speciﬁc conceptual and practical problems such as sample design, choice of index formula, adjustment of prices for quality changes, and the treatment of new products.

Statistical oﬃces must therefore rely on the underlying economic and statistical principles laid out in this manual to arrive at practical solutions.
The consumer price index The CPI is an index that measures the rate at which the prices of consumption goods and services are changing from month to month (or from quarter to quarter).

The prices are collected from shops or other retail outlets.

The usual method of calculation is to take an average of the period-to-period price changes for the diﬀerent products, using as weights the average amounts that households spend on them.
CPIs are oﬃcial statistics that are usually produced by national statistical oﬃces, ministries of labour or central banks.

They are published as quickly as possible, typically about ten days after the end of the most recent month or quarter.

The manual is intended for the beneﬁt of users of CPIs, as well as for the statistical agencies that compile the indices.

It is designed to do two things.

First, it explains in some detail the methods that are actually used to calculate a CPI.
Second, it explains the underlying economic and statistical theory on which the methods are based.

A CPI measures the rate of price inﬂation as experienced and perceived by households in their role as consumers.

It is also widely used as a proxy for a general index of inﬂation for the economy as a whole, partly because of the frequency and timeliness with which it is produced.

It has become a key statistic for purposes of economic policy-making, especially monetary policy.
It is often speciﬁed in legislation and in a wide variety of private contracts as the appropriate measure of inﬂation for the purposes of adjusting payments (such as wages, rents, interest and social security beneﬁts) for the eﬀects of inﬂation.

It can therefore have substantial and wide-ranging ﬁnancial implications for governments and businesses, as well as for households.
This manual provides guidelines for statistical oﬃces or other agencies responsible for constructing a CPI, bearing in mind that the resources available for this purpose are limited.

Calculating a CPI cannot be reduced to a simple set of rules or standard set of procedures that can be mechanically followed in all circumstances.
While there are certain general principles that may be universally applicable, the procedures followed in practice, whether they concern the collection or processing of the prices or the methods of aggregation, have to take account of particular circumstances.

These include the main use of the index, the nature of the markets and pricing practices within the country, and the resources available to the statistical oﬃce.

Sta- tistical oﬃces have to make choices.


The manual explains the underlying economic and statistical concepts and principles needed to enable statistical oﬃces to make their choices in eﬃcient and cost-eﬀective ways and to be aware of the full implications of their choices.

The manual draws upon the experience of many statistical oﬃces throughout the world.

The procedures they use are not static, but continue to evolve and improve in response to several factors.
First, research continually reﬁnes and strengthens the economic and statistical theory underpinning CPIs.

For example, clearer insights have recently been obtained on the relative strengths and weaknesses of the various formulae and methods used to process the basic price data collected for CPI purposes.

Second, recent advances in information and communications technology have aﬀected CPI methods.

Both of these theoretical and data developments can impinge on all the stages in compiling a CPI.
New technology can aﬀect the methods used to collect prices and transmit them to the central statistical oﬃce.

It can also improve the processing and checking, including the methods used to adjust prices for changes in the quality of the goods and services xix PREFACE covered.

Finally, improved formulae help in calculating more accurate and reliable higher-level indices, including the overall CPI itself.
International standards for CPIs Some international standards for economic statistics have evolved primarily in order to enable inter- nationally comparable statistics to be compiled.

However, individual countries also stand to beneﬁt from international standards.

The CPI standards described in this manual draw upon the collective experience and expertise accumulated in many countries.

All countries can beneﬁt by having easy access to this experience and expertise.
In many countries, CPIs were ﬁrst compiled mainly in order to be able to adjust wages to compensate for the loss of purchasing power caused by inﬂation.

Consequently, the responsibility for compiling CPIs was often entrusted to ministries, or departments, of labour.

The International Conference of Labour Statis- ticians (ICLS), convened by the Governing Body of the ILO, therefore provided the natural forum in which to discuss CPI methodology and develop guidelines.
The ﬁrst international standards for CPIs were promulgated in 1925 by the Second ICLS.

The ﬁrst set of standards referred to ‘‘cost of living’’ indices rather than CPIs.

A distinction is now drawn between two diﬀerent types of index.
A consumer price index can be deﬁned simply as measuring the change in the cost of purchasing a given ‘‘basket’’ of consumption goods and services, whereas a cost of living index is deﬁned as measuring the change in the cost of maintaining a given standard of living, or level of utility.

For this reason, the Tenth ICLS in 1962 decided to adopt the more general term ‘‘consumer price index’’, which should be understood to embrace both concepts.

There need be no conﬂict between the two.


As explained in the manual, the best-practice methods are likely to be very similar, whichever approach is adopted.

The international standards have been revised three times, in 1947, 1962 and 1987, in the form of reso- lutions adopted by the ICLS.

The 1987 standards on CPI were followed by a manual on methods (Turvey, 1989), which provided guidance to countries on the practical application of the standards.
The background to the present revision A few years after the publication of the 1989 ILO manual, it became clear that a number of outstanding and controversial methodological problems needed further investigation and analysis.

An expert group was formed consisting of specialists in price indices from national statistical oﬃces, international organizations and universities from around the world.
It met for the ﬁrst time in Ottawa in 1994, and became known as the ‘‘Ottawa Group’’, one of the city groups established by the United Nations Statistical Commission to address selected problems in statistical methods.

During the course of seven meetings of the Ottawa Group between 1994 and 2003, over 100 research papers on the theory and practice of price indices were presented and discussed.
One outcome was that it became apparent that existing CPI methods could be improved and strengthened in a number of ways.

At the same time, the control of inﬂation had become a high-priority policy objective in most countries.

Not only is the CPI widely used to measure and monitor inﬂation, but inﬂation targets in many countries are set speciﬁcally in terms of a precise rate of change in the CPI.
The slowing down of inﬂation in many parts of the world in the 1990s, as compared with the 1970s and 1980s, far from reducing interest in CPI methodology, actually stimulated a demand for more accurate, precise and reliable measures of inﬂation.

When the rate of inﬂation slows to only 2 or 3 per cent per year, even a small error or bias in the CPI becomes relatively signiﬁcant.
In order to be sure about the accuracy of CPIs, governments or research institutes in a few countries commissioned special groups of experts to investigate and evaluate the methods used.

The methodology used to calculate CPIs was subjected to public interest and scrutiny unknown in the past.

One conclusion reached was that existing methods might lead to some upward bias.
Many academic and government economists and other users of CPIs became convinced of this, believing that insuﬃcient allowance was being made for improvements in the quality of many goods and services.

In fact, the extent and sometimes even the direction of such bias are uncertain.

It will also, of course, vary between diﬀerent types of consumption goods and services, and its total eﬀect on the overall CPI will vary between countries.

However, the bias is potentially large.


For this reason, this manual addresses in some detail the issue of adjusting prices for changes in quality, drawing upon the most recent research in this area.

There are other sources of possible bias, such as that resulting from working with an out-of-date and unrepresentative basket of goods and services.

Bias may also result from the sampling and price collection methods used.
Several chapters deal with these issues, with an overall summary of possible errors and biases given in Chapter 11.

CPIs are widely used for the index linking of social beneﬁts such as pensions, unemployment beneﬁts and other government payments, and also as escalators for adjusting prices in long-term contracts.

The cumulative eﬀects of even a small bias could be substantial over the long term and could have considerable ﬁnancial consequences for government budgets.
Government agencies, especially ministries of ﬁnance, have xx PREFACE therefore taken a renewed interest in CPIs, examining their accuracy and reliability more closely and carefully than in the past.

In response to the various developments outlined above, the need to revise, update and expand the 1989 ILO manual was gradually recognized and accepted during the late 1990s.
A formal recommendation to revise the manual was made at the joint UNECE/ILO Meeting on Consumer Price Indices, held in Geneva at the end of 1997.

Responsibility for the revision was entrusted to the main international organizations interested in the measurement of inﬂation.

This strategy was endorsed in 1998 by the United Nations Statistical Com- mission, which also agreed to the conversion of the Ottawa Group into a formal Intersecretariat Working Group on Price Statistics (IWGPS).
The Sixteenth ICLS, meeting in 1998, also recommended that the Fourteenth ICLS resolution concerning consumer price indices, adopted in 1987, should be revised.

The preparation of the draft revised resolution discussed at the Seventeenth ICLS (24 November–3 December 2003) was carried out by the ILO Bureau of Statistics in parallel with the preparation of this revised manual.
Every eﬀort has been made to ensure that the two documents are consistent and mutually supportive.1 Some concerns about current index methods This new manual takes advantage of the wealth of new research on index number theory and methods in the last decade to address the kinds of concerns referred to above.

It recommends some new practices and its purpose is not simply to codify existing statistical agency practices.
It is useful to highlight a few of the main concerns that have led to many topics being dealt with in some depth in the manual.

The traditional standard methodology underlying a typical CPI is based on the concept of a Laspeyres price index.

A Laspeyres index measures the change between two periods of time in the total cost of purchasing a basket of goods and services that is representative of the ﬁrst, or base, period.
The base period basket of consumer purchases is priced ﬁrst at base period prices and then repeatedly priced at the prices of successive time periods.

This methodology has at least three practical advantages.

It is easily explained to the public; it can make repeated use of the same data on consumer purchases that date from some past household survey or administrative source (rather than requiring new data each month); and it need not be revised, assuming users are satisﬁed with the Laspeyres concept.
Another notable advantage is that the Laspeyres is consistent in aggregation down to the lowest level of aggregation.

The index can be broken down into sub-aggregates that are interrelated in a simple way.

Statistical agencies actually calculate their CPIs by implementing the Laspeyres index in its alterna- tive form as a weighted average of the observed price changes, or price relatives, using the base period expenditure shares as weights.
Unfortunately, although the Laspeyres is a simple concept, it is diﬃcult to calculate a proper Laspeyres index in practice.

Consequently, statistical agencies have to resort to approximations: It is generally impossible to obtain accurate expenditure shares for the base period at the level of individual commodities, so statistical agencies settle for getting base period expenditure weights at the level of 100–1,000 product groups.
For each of the chosen product groups, agencies collect a sample of representative prices from outlets rather than attempting to collect every single transaction price.

They use equally weighted (rather than expenditure-weighted) index formulae to aggregate these elementary product prices into an elementary aggregate index, which will in turn be used as the price relative for each of the 100–1,000 product groups when calculating the higher-level Laspeyres index.
It is recognized that this two-stage procedure is not entirely consistent with the Laspeyres methodology (which requires weighting at each stage of aggre- gation).

However, for a number of theoretical and practical reasons, statistical agencies judge the resulting elementary index price relatives to be suﬃciently accurate to insert into the Laspeyres formula at the higher stage of aggregation.
This methodology dates back to the work of Mitchell (1927) and Knibbs (1924), and other pioneers who introduced it 80 or 90 years ago, and it is still used today.

Although most statistical agencies have traditionally used the Laspeyres index as their target index, both economic and index number theory suggest that some other types of indices may be more appropriate target indices to aim at: namely, the Fisher, Walsh or To¨ rnqvist–Theil indices.
As is well known, the Laspeyres index has an upward bias compared with these target indices.

Of course, these target indices may not be achievable by a statistical agency, but it is necessary to have some sort of theoretical target to aim at.

Having a target concept is also necessary so that the index that is actually produced by a statistical agency can be evaluated to see how close it comes to the theoretical ideal.
In the theoretical chapters of the manual, four main approaches to index number theory are described: 1The 2003 resolution concerning consumer price indices is reproduced in Annex 3.

It can also be found on the ILO Bureau of Statistics web site: http://www.ilo.org/public/english/bureau/stat.
xxi PREFACE (1) ﬁxed basket approaches and symmetric averages of ﬁxed baskets; (2) the stochastic (statistical estimator) approach to index number theory; (3) test (axiomatic) approaches; and (4) the economic approach.

Approaches (3) and (4) will be familiar to many price statisticians and expert users, but perhaps a few words about approaches (1) and (2) are in order.

The Laspeyres index is an example of a basket index.
The concern from a theoretical point of view is that there is an equally valid alternative for the two periods being compared: the Paasche index, which uses the basket of quantities from the current period.

If there are two equally valid estimators for the same concept, then statistical theory suggests taking an average of the two.

However, there is more than one kind of average and the question of which average to take is not trivial.
The manual proposes that the ‘‘best’’ average is the geometric average of the Laspeyres and Paasche indices (the Fisher ideal).

Alternatively, the ‘‘best’’ basket is one whose quantities are geometric averages of the quantities in both periods (the Walsh index).

From the statistical estimation perspective, the ‘‘best’’ index number is a geometric average of the price relatives that uses the (arithmetic) average expenditure share in two periods as weights (the To¨ rnqvist– Theil index).
One additional result from index number theory should be mentioned here: the problem of deﬁning the price and quantity of a product that should be used for each period in the index number formula.

The problem is that the same product may be sold at a number of diﬀerent prices.

So the question arises, what price would be most representative of the sales of this product for the period?
The answer is the unit value, since this price multiplied by the total quantity sold during the period equals the value of sales.

Of course, the manual does not endorse taking unit values over heterogeneous products; unit values should only be calculated for identical products.

Six main areas of concern with the standard methodology are listed below.

They are not ranked in order of importance, and all are considered to be important: 1.
At the ﬁnal stage of aggregation, a conventional CPI is not a true Laspeyres index since the expenditure weights pertain to a reference base year that is diﬀerent from the base month (or quarter) for prices.

Thus, the expenditure weights are annual whereas the prices are collected monthly.

To be a true Laspeyres index, the period that provides the expenditure weights must coincide with the reference period for the prices.
In fact, the index actually calculated by many statistical agencies at the last stage of aggregation has a weight reference period that precedes the base price period.

Indices of this type are likely to have some upward bias compared to a true Laspeyres index, especially if the expenditure weights are price- updated from the weight reference period to the Laspeyres base period.
It follows that they must have deﬁnite upward biases compared to theoretical target indices such as the Fisher, Walsh or To¨ rnqvist– Theil indices.

2.

At the early stages of aggregation, unweighted averages of prices or price relatives are used.

Until recently, when scanner data from electronic points of sale became more readily available, it was thought that the biases that might result from the use of unweighted indices were not particularly signiﬁcant.
However, recent evidence suggests that there is potential for signiﬁcant upward bias at lower levels of aggregation compared to results that are generated by the preferred target indices mentioned above.

3.
3.

The third major concern with standard CPI methodology is that, although statistical agencies generally recognize that there is a problem with the treatment of quality change and new goods, it is diﬃcult to work out a coherent methodology for these problems in the context of a Laspeyres index that uses a ﬁxed set of quantities.
The most widely received good practice in quality adjusting price indices is ‘‘hedonic regression’’, which characterizes the price of a product at any given time as a function of its physical and economic characteristics as compared with substitutes.

In fact, there is a considerable amount of con- troversy on how to integrate hedonic regression methodology into the CPI’s theoretical framework.
Both the theoretical and the more practically oriented chapters in the manual devote a lot of attention to these methodological issues.

The problems created by the disappearance of old, and the appearance of new, products are now much more severe than they were when the traditional CPI methodology was developed some 80 years ago (when the problem was mostly ignored).
For many categories of products, such as models of consumer durables, those priced at the beginning of the year are simply no longer available by the end of the year.

Sample attrition creates tremendous methodological problems.

At lower levels of aggregation, it becomes necessary (at least in many product groups) to use chained indices rather than ﬁxed base indices.

Certain unweighted indices are liable to have substantial bias when chained.

4.
4.

A fourth major area of concern is related to the ﬁrst: that is, the treatment of seasonal commodities.

The use of annual quantities or annual expenditure shares is justiﬁed to a certain extent if one is interested in the longer-run trend of price changes.

However, some users, such as central banks, focus on short-term, month-to-month changes, in which case the use of annual weights can lead to misleading signals.
Monthly price changes for products that are out of season (i.e., the seasonal weights for the product class are small xxii PREFACE for those months) can be greatly magniﬁed by the use of annual weights.

The problem is worse when the products are not available at all at certain months of the year.
There are solutions to these seasonality problems, but they may not appeal to many CPI compilers and users since they involve the construction of two indices: one for the short-term measurement of price changes and another (more accurate) longer- term index that is adjusted for seasonal inﬂuences.

5.
5.

A ﬁfth concern with standard CPI methodology is that, in common with most economic statistics, services have been comparatively neglected in CPIs, notwithstanding the fact that they have become extremely important.

A typical CPI will collect many more goods prices than services prices and will have many more product groups for goods rather than services.
Traditionally, there has not been much focus on the problems involved in measuring price and quantity changes for services, even though they raise serious conceptual and practical problems.

Some examples of diﬃcult-to-measure services are: insurance, gambling, ﬁnancial services, advertising, telecommunications, entertainment and housing services.
In many cases, statistical agencies simply do not have the resources or methodologies at their disposal to deal adequately with these diﬃcult measurement problems.

6.

A ﬁnal concern with existing CPI methodology is that it tends not to recognize that more than one CPI may be required to meet the needs of diﬀerent users.

For example, some users may require information on the month-to-month movement of prices in a timely fashion.
This requires a basket index with predetermined (even though possibly inappropriate and out-of-date) weights that are instantly available.

However, other users may be more interested in a more accurate or representative measure of price change and may be willing to sacriﬁce timeliness for increased accuracy.

For this reason, the United States Bureau of Labor Statistics provides, on a retrospective basis, a superlative index that uses both current and base period weight information in a symmetrical way.
This is an entirely reasonable development, recognizing that diﬀerent users have diﬀerent needs.

A second example where more than one index might be compiled relates to owner-occupied housing.

Good cases have been made for three diﬀerent treatments: the acquisitions approach, the rental equivalence approach and the user cost approach.

However, these three approaches may give quite diﬀerent numerical results in the short run.
A statistical agency has to opt for one approach, but since all three command support, indices using the other two approaches could be made available as analytical series for interested users.

A third example of where more than one index would be useful occurs when, because of seasonal commodities, the month-to-month index may not be based on the same set of products as one that compares the month with the same month a year earlier.

The above kinds of concern are addressed in this manual.
Frank discussions of these matters should stimulate the interest of professional economists and statisticians in universities, government departments, central banks, and so on, to address these measurement problems and to provide new solutions that can be used by statistical agencies.

Public awareness of these areas should also heighten awareness of the need for additional resources to be allocated to statistical agencies so that economic measurement will be improved.
The Harmonized Indices of Consumer Prices Within the European Union (EU), the convergence of inﬂation in Member States was an important prerequisite for the formation of a monetary union in 1999.

This required a precisely deﬁned measure of inﬂation and an agreed methodology to ensure that the diﬀerent countries’ price indices are comparable.
A detailed and systematic review of all aspects of the compilation of CPIs was therefore undertaken during the 1990s by all the national statistical oﬃces of the EU Member States in collaboration with Eurostat, the Statistical Oﬃce of the EU.

This work culminated in the elaboration of a new EU standard for the 29 Member and candidate States, and led to the development of the EU’s Harmonized Indices of Consumer Prices (HICPs).

A summary of HICP methodology is given in Annex 1 to this manual.
Work on the HICPs proceeded in parallel with that of the IWGPS, many of whose experts also parti- cipated both in work on the HICPs and in the present revision of this manual.

Although the methodology elaborated here has much in common with that adopted for the HICPs, there are also diﬀerences.
The HICPs were developed for a very speciﬁc purpose, whereas the methodology developed in this manual is intended to be ﬂexible, multi-purpose and applicable to all countries, whatever their economic circum- stances and level of development.

The manual also provides considerably more detail, information, explanation and rationalization of CPI methodology and the associated economic and statistical theory than is to be found in the HICP standards.
The organization of the revision The six international organizations listed at the beginning of this preface, concerned with both the measurement of inﬂation and policies designed to control it, have collaborated on the revision of this manual.

They have provided, and continue to provide, technical assistance on CPIs to countries at all levels xxiii PREFACE of development, including those in transition from planned to market economies.
They joined forces for the revision of this manual, establishing the IWGPS for the purpose.

The role of the IWGPS was to organize and manage the process rather than act as an expert group.
The responsibilities of the IWGPS were to: appoint the various experts on price indices who participated in the revision process, either as members of the Technical Expert Group (TEG/CPI), providing substantive advice on the content of the manual, or as authors; provide the ﬁnancial and other resources needed; arrange meetings of the TEG/CPI, prepare the agendas and write the reports of the meetings; and arrange for the publication and dissemination of the manual.
Members of the IWGPS were also members of the TEG/CPI.

It is important to note that the experts participating in the TEG/CPI were invited in their personal capacity as experts and not as representatives, or delegates, of the national statistical oﬃces or other agencies in which they might be employed.

Participants were able to give their expert opinions without in any way committing the oﬃces from which they came.
The revision of the manual took ﬁve years, and involved multiple activities: the development of the manual outline and the recruitment of experts to draft the various chapters; the review of the draft chapters by the members of the TEG/CPI, the IWGPS and other experts; the posting of the draft chapters on a special web site for comment by interested individuals and organizations; discussions by a small group of experts from statistical agencies and universities on the ﬁnalization of all the chapters; ﬁnal copy-editing of the whole manual.
Links with the Producer price index manual One of the ﬁrst decisions of the IWGPS was that a new international manual on producer price indices (PPIs) should be produced simultaneously with this manual.

Whereas there have been international standards for CPIs for over 70 years, the ﬁrst international manual on producer price indices was not produced until 1979 (United Nations, 1979).
Despite the importance of PPIs for measuring and analysing inﬂation, the methods used to compile them have been comparatively neglected, at both national and international levels.

A new Producer price index manual (ILO, IMF, OECD, Eurostat, UNECE and the World Bank, forth- coming) has therefore been developed and written in parallel with this manual.

The IWGPS established a second Technical Expert Group on PPIs whose membership overlapped with that of the Technical Expert Group on CPIs.
The two groups worked in close liaison with each other.

The methodologies of PPIs and CPIs have much in common.

Both are based on essentially the same underlying economic and statistical theory, except that the CPI draws on the economic theory of consumer behaviour whereas the PPI draws on the economic theory of production.

However, the two economic theories are isomorphic and lead to the same kinds of conclusions with regard to index number compilation.
The two manuals have similar contents and are fully consistent with each other conceptually, sharing common text when appropriate.

Most members of the Technical Expert Groups on CPIs and PPIs also participated as active members of the Ottawa Group.

The two manuals were able to draw upon the contents and conclusions of all the numerous papers presented at meetings of the Group.
xxiv ACKNOWLEDGEMENTS The organizations of the IWGPS wish to thank all those involved in the drafting and production of the manual.

Particular thanks go to Peter Hill, the editor, W. Erwin Diewert, who contributed extensively to the theoretical chapters of the manual, and Bert Balk, who acted as referee for all the theoretical chapters.

Their eﬀorts greatly enhanced the quality of the manual.
The authors of the chapters are as follows: Jorgen Dale´ n, A. Sylvester Young and Bert Balk Preface Peter Hill, Paul Armknecht and W. Erwin Diewert Reader’s guide Peter Hill An introduction to consumer price index methodology Peter Hill Uses of consumer price indices Peter Hill Concepts and scope Peter Hill and Fenella Maitland-Smith Expenditure weights and their sources Valentina Stoevska and Carsten Boldsen Sampling Price collection David Fenwick Adjusting for quality change Mick Silver Item substitution, sample space and new products Mick Silver Calculating consumer price indices in practice Carsten Boldsen and Peter Hill Some special cases Keith Woolford, David Fenwick, contributors from several statistical oﬃces Errors and bias John Greenlees and Bert Balk Organization and management David Fenwick Publication, dissemination and user relations Tom Griﬃn The system of price statistics Kimberly Zieschang Basic index number theory W. Erwin Diewert The axiomatic and stochastic approaches to index number theory W. Erwin Diewert The economic approach to index number theory: The single-household case W. Erwin Diewert The economic approach to index number theory: The many-household case W. Erwin Diewert Price indices using an artiﬁcial data set W. Erwin Diewert Elementary indices W. Erwin Diewert Quality change and hedonics Mick Silver The treatment of seasonal products W. Erwin Diewert Durables and user costs W. Erwin Diewert A glossary of main terms and annex to the glossary Peter Hill and Bert Balk 6
the glossary Peter Hill and Bert Balk Annexes 1 Harmonized Indices of Consumer Prices (European Union) Alexandre Makaronidis, Keith Hayes 2 3 Classiﬁcation of Individual Consumption according to Purpose (COICOP)-Extract United Nations Resolution concerning consumer price indices adopted by the Seventeenth International Conference of Labour Statisticians, 2003 ILO 4 Spatial comparisons of consumer prices, purchasing power parities and the International Comparison Program Prasada Rao xxv
 ACKNOWLEDGEMENTS The aﬃliations of the authors are as follows: Bert Balk Carsten Boldsen Jorgen Dale´ n W. Erwin Diewert David Fenwick John Greenlees Tom Griﬃn Keith Hayes Peter Hill Fenella Maitland-Smith Alexandre Makaronidis Prasada Rao Mick Silver Valentina Stoevska Keith Woolford A. Sylvester Young Kimberly Zieschang Statistics Netherlands Statistics Denmark Expert University of British Columbia, Canada United Kingdom Oﬃce of National Statistics (ONS) United States Bureau of Labor Statistics (BLS) Expert Eurostat Expert, manual editor OECD Eurostat University of Queensland, Australia Cardiﬀ University, United Kingdom ILO Australian Bureau of Statistics (ABS) ILO IMF The manual has also beneﬁted from valuable contributions by many other experts, including: Martin Boon (Statistics Netherlands); Heber Camelo and Ernestina Pe´ rez (Economic Commission for Latin America and the Caribbean); Denis Fixler (United States Bureau of Economic Analysis); Leendert Hoven (Statistics Netherlands); Michel Mouyelo-Katoula (African Development Bank); Carl Obst (formerly OECD); Bou- chaib Thich (De´ partement de la pre´ vision e´ conomique et du plan, Morocco); and Ralph Turvey (expert).
The following also gave helpful advice and comments: Statistics Austria; Statistics Singapore; United States BLS; Michael Anderson (ABS); Rob Edwards (ABS); Eivind Hoﬀmann (ILO); participants at the International Workshop on Consumer Price Indices, Singapore, June 2001; and the members of the Ottawa Group.

The IWGPS established the Technical Expert Group on the CPI (TEG/CPI) for the revision of the manual.
Members of the IWGPS were also members of the TEG/CPI, whose individual members were: David Fenwick Paul Armknecht John Astin* Bert Balk W. Erwin Diewert Yoel Finkel Carsten Boldsen John Greenlees Paul Haschka Peter Hill Jean-Claude Roman* Bohdan Schultz* Mick Silver Kimberly Zieschang Chair, United Kingdom ONS TEG/PPI chair, IMF Eurostat Statistics Netherlands University of British Columbia, Canada Israel Central Bureau of Statistics Statistics Denmark United States BLS Statistics Austria Editor Eurostat Statistics Canada Cardiﬀ University, United Kingdom IMF The UNECE (Jan Karlsson, Lidia Bratanova*, Miodrag Pesut*, Tihomira Dimova*) and the ILO (Valentina Stoevska) jointly acted as the Secretariat of the TEG/CPI.
The TEG/CPI met seven times: 11–12 February 1999 (Geneva), 2 November 1999 (Geneva), 5–6 February 2001 (Washington, DC), 25–26 June 2001 (Geneva), 31 October 2001 (Geneva), 19–21 March 2002 (Lon- don) and 14–15 October 2002 (London).

The IWGPS met formally ﬁve times: 24 September 1998 (Paris), 11 February 1999 (Geneva), 2 November 1999 (Geneva), 21–22 March 2002 (London) and 5 December 2003 (Geneva).

A number of informal meetings were also held.


The ILO was the Secretariat of the Group and A. Sylvester Young the chairperson of the IWGPS.

During the revision of the manual, the CPI manual editor (Peter Hill), the TEG-CPI chairperson (David Fenwick), the PPI manual editor and the TEG/PPI chairperson (Paul Armknecht) participated in the meetings of the IWGPS.

* These members served for only part of the period.
xxvi The ﬁnal publication of the English version of this manual was coordinated, with the involvement of the IWGPS member organizations, by Valentina Stoevska of the ILO Bureau of Statistics.

The ILO Bureau of Publications provided extensive editorial and support services for the production process.

We should also like to thank Angela Haden and Barbara Campanini for their thorough copy-editing of the ﬁnal draft.
ACKNOWLEDGEMENTS xxvii READER’S GUIDE International manuals in the ﬁeld of economic statistics have traditionally been intended to provide guidance about concepts, deﬁnitions, classiﬁcations, coverage, valuation, the recording of data, aggregation pro- cedures, formulae, and so on.

They have been intended mainly to assist compilers of the relevant statistics in individual countries.

This manual has the same principal objective.


The manual is also intended for the beneﬁt of users of consumer price indices (CPIs), such as government and academic economists, ﬁnancial experts and other informed users.

The CPI is a key statistic for policy purposes.

It attracts a great deal of attention from the media, governments and the public at large in most countries.
Despite its apparent simplicity, the CPI is a sophisticated concept that draws upon a great deal of economic and statistical theory, and requires complex data manipulation.

This manual is therefore also intended to promote greater understanding of the properties of CPIs.

In general, compilers and users of economic statistics must have a clear perception of what the statistics are supposed to measure, in principle.

Measurement without theory is unacceptable in economics, as in other disciplines.
The manual therefore contains a thorough, comprehensive and up-to-date survey of the relevant economic and statistical theory.

This makes the manual completely self-contained on both the theory and practice of CPI measurement.

The resulting manual is large.

As diﬀerent readers may have diﬀerent interests and priorities, it is not possible to devise a sequence of chapters that suits everyone.
Indeed, because this manual is intended to serve as a reference source, it will not necessarily be read from cover to cover.

Many readers may be interested in only a selection of chapters.

The purpose of this reader’s guide is to provide a map of the contents of the manual that will assist readers with diﬀerent interests and priorities.

An overview of the sequence of chapters Chapter 1 is a general introduction to CPI methodology, and is intended for all readers.
It provides the basic information needed to understand the subsequent chapters.

It summarizes index number theory, as explained in detail in Chapters 15 to 23, and outlines the main steps involved in the actual compilation of a CPI, drawing on material in Chapters 3 to 9.

It does not provide a summary of the manual as whole, as it does not cover some speciﬁc topics and special cases that are not of general relevance.
Chapter 2 explains how CPIs have evolved in response to the demands made upon them and how the uses of CPIs aﬀect the choice of methodology to be used.

Chapter 3 is concerned with a number of basic concepts, principles and classiﬁcations, as well as with the scope or coverage of an index.

The scope of a CPI can vary signiﬁcantly from country to country.
Chapters 4 to 9 form an interrelated sequence describing the various steps involved in the compilation of a CPI from the collection and processing of the price data through to the calculation of the ﬁnal index.

Chapter 4 explains how the expenditure weights attached to the price changes for diﬀerent goods and services are derived.

These weights are typically based on household expenditure surveys supplemented by data from other sources.

Chapter 5 deals with sampling issues.


A CPI is essentially an estimate based on a sample of prices.

Chapter 5 considers sampling design, and the pros and cons of random versus purposive sampling.

Chapter 6 is devoted to the procedures actually used to collect the prices from a selection of retail outlets or other suppliers.

It deals with topics such as questionnaire design, the speciﬁcation of the items selected, the use of scanner data and the use of hand-held computers.
Chapter 7 addresses the diﬃcult question of how to adjust prices for changes over time in the quality of the goods or services selected.

Changes in value resulting from changes in quality count as changes in quantity, not price.

Disentangling the eﬀects of quality change poses serious theoretical and practical problems for compilers.

Chapter 8 covers the closely related question of how to deal with new goods or services not previously purchased and for which there are no prices in earlier periods.
Chapter 9 pulls together the material contained in the preceding ﬁve chapters and gives a step-by-step summary of the various stages of CPI calculation.

It describes both the elementary price indices calculated from the raw prices collected for small groups of products and the subsequent averaging of the elementary indices to obtain indices at higher levels of aggregation up to the overall CPI itself.
Chapter 10 deals with a number of cases that require special treatment: for example, goods and services for which prices are not quoted separately, being embedded within composite transactions covering more xxix READER’S GUIDE than one item.

It also examines the case of owner-occupied housing.

Chapter 11 considers the errors and biases to which CPIs may be subject.

Chapter 12 deals with issues of organization and management.
Conducting the price surveys and pro- cessing the results is a massive operation that requires careful planning and organization, and also eﬃcient management.

Chapter 13 is concerned with the publication or dissemination of the results.

Chapter 14 marks a break in the sequence of chapters, as it not concerned with the compilation of a CPI.

Its purpose is diﬀerent, namely to examine the place of the CPI in the general system of price statistics.
The CPI should not be treated as an independent, isolated statistic.

The ﬂow of consumer goods and services to which it relates is itself only one of a set of interdependent ﬂows within the economy as a whole.

The analysis of inﬂation requires more than one index, and it is essential to know exactly how the CPI relates to the producer price index (PPI) and to other price indices, such as indices of export and import prices.
The supply and use matrix of the System of National Acounts provides the appropriate conceptual framework within which to examine these interrelationships.

Chapters 15 to 18 provide a systematic and detailed exposition of the index number and economic theory underlying CPIs.

Five diﬀerent approaches to index number theory are examined that between them cover all aspects of index number theory.
Collectively, they provide a comprehensive and up-to-date survey of index number theory, including recent methodological developments as reported in journals and conference proceedings.

Chapter 15 provides an introduction to index number theory focusing on the decomposition of value changes into their price and quantity components.

Chapter 16 examines the axiomatic and stochastic approaches to CPIs.
The axiomatic, or test, approach lists a number of properties that it is desirable for index numbers to possess and tests speciﬁc formulae to see whether or not they possess them.

Chapter 17 explains the economic approach based on the economic theory of consumer behaviour.

On this approach, a CPI is deﬁned as a cost of living index (COLI).

Although COLIs cannot be calculated directly, a certain class of index numbers, known as superlative indices, can be expected to approximate COLIs in practice.
An increasing number of economists and other users have concluded that, in principle, the preferred, ideal index for CPI purposes should be a superlative index, such as the Fisher index.

This is reinforced by the fact that the Fisher also emerges as a very desirable index on axiomatic grounds.

Chapter 18 deals with aggregation issues.

Chapter 19 uses a constructed data set to illustrate the numerical consequences of using diﬀerent index number formulae.
It demonstrates that, in general, the choice of index number formula can make a considerable diﬀerence, but that diﬀerent superlative indices all tend to approximate each other.

Chapter 20 addresses the important question of what is the theoretically most appropriate form of elementary price index to calculate at the ﬁrst stage of CPI compilation when no information is available on quantities or expenditures.
This has been a comparatively neglected topic until recently, even though the choice of formula for an elementary index can have a signiﬁcant impact on the overall CPI.

The elementary indices are the basic building blocks from which CPIs are constructed.

Chapters 21 to 23 deal with diﬃcult issues.

Chapter 21 discusses adjusting for quality change, including the hedonic approach, from a theoretical viewpoint.

Chapter 22 examines the treatment of seasonal pro- ducts.
Finally, Chapter 23 considers the treatment of durable goods.

There is some tension in both national accounts and CPIs resulting from the fact that owner-occupied houses are treated as assets, whereas consumer durables are not.

These treatments are not easy to reconcile conceptually and Chapter 23 dis- cusses the theoretical issues involved.
The manual concludes with a glossary of terms, a bibliography, and four annexes on the following topics: the Harmonized Indices of Consumer Prices (HICPs) of the European Union; the Classiﬁcation of Individual Consumption according to Purpose (COICOP), a household expenditure classiﬁcation; the resolution concerning consumer prices indices adopted by the Seventeenth International Conference of Labour Statisticians, 2003; spatial comparisons of consumer prices, using purchasing power parities and the International Com- parison Program.
Suggested reading plans Diﬀerent readers may have diﬀerent needs and priorities.

Readers interested mainly in the compilation of CPIs may not wish to pursue all the ﬁner points of the underlying economic and statistical theory.

Con- versely, readers interested more in the use of CPIs for analytic or policy purposes may not be so interested in reading about the technicalities of conducting and managing price surveys.
Not all readers will want to read the entire manual but all readers, whatever their preferences, will ﬁnd it useful to read the ﬁrst three chapters.

Chapter 1 provides a general introduction to the whole subject by giving an overview of the CPI theory and practice that is presented in the manual.

It covers the basic xxx READER’S GUIDE knowledge required for understanding subsequent chapters.

Chapter 2 explains why CPIs are calculated and what they are used for.
Chapter 3 examines a number of fundamental concepts and the scope of a CPI.

A reading plan for compilers Chapters 4 to 13 are primarily for compilers.

They follow a logical sequence that roughly matches the various stages of the actual compilation of a CPI, starting with the derivation of the expenditure weights and the collection of the price data, and ﬁnishing with the publication of the ﬁnal index.

Chapter 14 is intended equally for compilers and users of CPIs.
It places CPIs in perspective within the overall system of price indices.

The remaining chapters from 15 to 23 are mainly theoretical.

Compilers may ﬁnd it necessary to pursue certain theoretical topics in greater depth, in which case they have immediate access to the relevant material.

It would be desirable for compilers to be acquainted with at least the basic index number theory set out in Chapter 15 and the numerical example developed in Chapter 19.
The material in Chapter 20 on elementary price indices is also particularly important for compilers.

A reading plan for users Although all readers will ﬁnd Chapters 1 to 3 useful, the subsequent ten chapters are designed primarily for compilers.

Two topics that have, however, aroused considerable interest among many users are the treatment of quality change and new products.

These are discussed at some length in Chapters 7 and 8.
Users may also ﬁnd Chapter 9 particularly helpful as it provides a concise description of the various stages of compiling a CPI.

Chapter 11 on errors and bias, and Chapter 14 on the system of price statistics are also of equal interest to users and compilers.

Chapters 15 to 23 covering the underlying economic and statistical theory are likely to be of interest to many users, especially professional economists and students of economics.
References In the past, international manuals on economic statistics have not usually provided references to the associated literature.

It was not considered helpful to cite references when the literature was mostly conﬁned to printed volumes, including academic journals or proceedings of conferences, located only in university or major libraries.

Compilers working in many statistical oﬃces were unlikely to have ready access to such literature.
This situation has been completely transformed by the Internet and the Web, which make all such literature readily accessible.

Accordingly, this manual breaks with past tradition by including a compre- hensive bibliography on the very large literature that exists on index number theory and practice.

xxxi AN INTRODUCTION TO CONSUMER PRICE INDEX METHODOLOGY 1 1.1 A price index is a measure of the proportionate, or percentage, changes in a set of prices over time.
A consumer price index (CPI) measures changes in the prices of goods and services that households consume.

Such changes aﬀect the real purchasing power of con- sumers’ incomes and their welfare.

As the prices of dif- ferent goods and services do not all change at the same rate, a price index can only reﬂect their average move- ment.
A price index is typically assigned a value of unity, or 100, in some reference period and the values of the index for other periods of time are intended to indicate the average proportionate, or percentage, change in prices from this price reference period.

Price indices can also be used to measure diﬀerences in price levels between diﬀerent cities, regions or countries at the same point in time.
1.2 Much of this manual and the associated eco- nomic literature on price indices is concerned with two basic questions: Exactly what set of prices should be covered by the index?

 What is the most appropriate way in which to average their movements?

These two questions are addressed in the early sections of this introduction.
1.3 Consumer price indices (CPIs) are index num- bers that measure changes in the prices of goods and services purchased or otherwise acquired by households, which households use directly, or indirectly, to satisfy their own needs and wants.

Consumer price indices can be intended to measure either the rate of price inﬂation as perceived by households, or changes in their cost of living (that is, changes in the amounts that the house- holds need to spend in order to maintain their standard of living).
There need be no conﬂict between these two objectives.

In practice, most CPIs are calculated as weighted averages of the percentage price changes for a speciﬁed set, or ‘‘basket’’, of consumer products, the weights reﬂecting their relative importance in household consumption in some period.

Much depends on how appropriate and timely the weights are.

1.4 This chapter provides a general introduction to, and overview of, the methodology for compiling CPIs.
It provides a summary of the relevant theory and practice of index number compilation that is intended to facilitate the reading and understanding of the detailed chapters that follow, some of which are inevi- tably quite technical.
It describes all the various steps involved in CPI compilation starting with the basic concept, deﬁnition and purpose of a CPI, followed by the sampling procedures and survey methods used to collect and process the price data, and ﬁnishing with a summary of the actual calculation of the index and its dissemination.
1.5 An introductory presentation of CPI methodol- ogy has to start with the basic concept of a CPI and the underlying index number theory, including the proper- ties and behaviour of the various kinds of index number that are, or might be, used for CPI purposes.

In prin- ciple, it is necessary to settle what type of index to cal- culate before going on to consider the best way in which to estimate it in practice, taking account of the resources available.
1.6 The main topics covered in this chapter are as follows: – the origins and uses of CPIs; – basic index number theory, including the axiomatic and economic approaches to CPIs; – elementary price indices and aggregate CPIs; – the transactions, activities and households covered by CPIs; – the collection and processing of the prices, including adjusting for quality change; – the actual calculation of the CPI; – potential errors and bias; – organization, management and dissemination policy.
In contrast, in this manual, the chapters dealing with index theory come later on; thus the presentation in this chapter does not follow the same order as the corre- sponding chapters of the manual.

1.7 It is not the purpose of this introduction to provide a complete summary of the contents of the manual.

The objective is rather to provide a short pre- sentation of the core methodological issues with which readers need to be acquainted before tackling the detailed chapters that follow.
Some special topics, such as the treatment of certain individual products whose prices cannot be directly observed, are not considered here as they are not central to CPI methodology.

The origins and uses of consumer price indices 1.8 CPIs must serve a purpose.

The precise way in which they are deﬁned and constructed depends very much on what they are meant to be used for, and by whom.

As explained in Chapter 15, CPIs have a long history dating back to the eighteenth century.
Laspeyres and Paasche indices, which are still widely used today, were ﬁrst proposed in the 1870s.

They are explained 1 CONSUMER PRICE INDEX MANUAL: THEORY AND PRACTICE below.

The concept of the cost of living index was introduced early in the twentieth century.

1.9 Traditionally, one of the main reasons for com- piling a CPI was to compensate wage-earners for inﬂa- tion by adjusting their wage rates in proportion to the percentage change in the CPI, a procedure known as indexation.
For this reason, oﬃcial CPIs tended to be- come the responsibility of ministries of labour, but most are now compiled by national statistical oﬃces.

A CPI that is speciﬁcally intended to be used to index wages is known as a compensation index.

1.10 CPIs have three important characteristics.

They are published frequently, usually every month but some- times every quarter.

They are available quickly, usually about two weeks after the end of the month or quarter.

They are also usually not revised.


CPIs tend to be closely monitored and attract a lot of publicity.

1.11 As CPIs provide timely information about the rate of inﬂation, they have also come to be used for a wide variety of purposes in addition to indexing wages.

For example: CPIs are widely used to index pensions and social security beneﬁts.

 CPIs are also used to index other payments, such as interest payments or rents, or the prices of bonds.
CPIs are also commonly used as a proxy for the gen- eral rate of inﬂation, even though they measure only consumer inﬂation.

They are used by some govern- ments or central banks to set inﬂation targets for purposes of monetary policy.
The price data collected for CPI purposes can also be used to compile other indices, such as the price indices used to deﬂate household consumption expenditures in national accounts, or the purchasing power parities used to compare real levels of consumption in diﬀer- ent countries.

1.12 These varied uses can create conﬂicts of inter- est.
For example, using a CPI as an indicator of general inﬂation may create pressure to extend its coverage to include elements that are not goods and services con- sumed by households, thereby changing the nature and concept of the CPI.

It should also be noted that because of the widespread use of CPIs to index a wide variety of payments – not just wages, but social security beneﬁts, interest payments, private contracts, etc.
– extremely large sums of money turn on their movements, enough to have a signiﬁcant impact on the state of government ﬁnances.

Thus, small diﬀerences in the movements of CPIs resulting from the use of slightly diﬀerent formulae or methods can have considerable ﬁnancial implications.

CPI methodology is important in practice and not just in theory.
be no single formula that would be preferred in all cir- cumstances, most economists and compilers of CPIs seem to be agreed that, in principle, the index formula should belong to a small class of indices called super- lative indices.

A superlative index may be expected to provide an approximation to a cost of living index.

A characteristic feature of a superlative index is that it treats the prices and quantities in both periods being compared symmetrically.
Diﬀerent superlative indices tend to have similar properties, yield similar results and behave in very similar ways.

Because of their properties of symmetry, some kind of superlative index is also likely to be seen as desirable, even when the CPI is not meant to be a cost of living index.
1.14 When a monthly or quarterly CPI is ﬁrst pub- lished, however, it is invariably the case that there is not suﬃcient information on the quantities and expenditures in the current period to make it possible to calculate a symmetric, or superlative, index.

While it is necessary to resort to second-best alternatives in practice, being able to make a rational choice between the various possibil- ities means having a clear idea of the target index that would be preferred in principle.
The target index can have a considerable inﬂuence on practical matters such as the frequency with which the weights used in the index should be updated.

1.15 A comprehensive, detailed, rigorous and up-to- date discussion of the relevant index number theory is provided in Chapters 15 to 23 of the manual.

The fol- lowing sections provide a summary of this material.
Proofs of the various propositions or theorems stated in this chapter are to be found in the later chapters, to which the reader may refer for further explanation.

Price indices based on baskets of goods and services 1.16 The purpose of an index number may be explained as comparing the values of households’ expen- ditures on consumer goods and services in two time periods.
Knowing that expenditures have increased by 5 per cent is not very informative if we do not know how much of this change is attributable to changes in the prices of the goods and services, and how much to changes in the quantities purchased.

The purpose of an index number is to decompose proportionate or per- centage changes in value aggregates into their overall components of price and quantity change.

A CPI is intended to measure the price component of the change in households’ consumption expenditures.
One way to do this is to measure the change in the value of an aggregate, holding the quantities constant.

Choice of index number 1.13 The ﬁrst question is to decide on the kind of index number to use.

The extensive references dealing with index theory in the bibliography reﬂect the fact that there is a very large literature on this subject.

Many diﬀerent kinds of mathematical formulae have been proposed over the past two centuries.
While there may Lowe indices 1.17 One very wide, and popular, class of price indices is obtained by deﬁning the index as the percent- age change, between the periods compared, in the total cost of purchasing a given set of quantities, generally described as a ‘‘basket’’.

The meaning of such an index is easy to grasp and to explain to users.
This class of index is called a Lowe index in this manual, after the 2 AN INTRODUCTION TO CONSUMER PRICE INDEX METHODOLOGY index number pioneer who ﬁrst proposed it in 1823 (see Chapter 15).

Most statistical oﬃces make use of some kind of Lowe index in practice.
1.18 Let there be n products in the basket with prices pi and quantities qi, and let the two periods compared be 0 and t. The Lowe index, PLo, is deﬁned as follows: pt iqi n P i=1 P p0 i qi PLo 1.19 In principle, any set of quantities could serve as the basket.

The basket does not have to be restricted to the quantities purchased in one or other of the two peri- ods compared, or indeed any actual period of time.
The quantities could, for example, be arithmetic or geometric averages of the quantities in the two periods.

For prac- tical reasons, the basket of quantities used for CPI pur- poses usually has to be based on a survey of household consumption expenditures conducted in an earlier period than either of the two periods whose prices are com- pared.
For example, a monthly CPI may run from Jan- uary 2000 onwards, with January 2000=100, but the quantities may be derived from an annual expenditure survey made in 1997 or 1998, or even spanning both those years.

As it takes a long time to collect and process expenditure data, there is usually a considerable time lag before such data can be introduced into the calculation of CPIs.

The basket may also refer to a year, whereas the index may be compiled monthly or quarterly.
1.20 The period whose quantities are actually used in a CPI is described as the weight reference period and it will be denoted as period b.

Period 0 is the price refer- ence period.
As just noted, b is likely to precede 0, at least when the index is ﬁrst published, and this is assumed here, but b could be any period, including one between 0 and t, if the index is calculated some time after t. The Lowe index using the quantities of period b can be written as follows: PLo n P i=1 n P i=1 iqb pt i i qb p0 i n P i=1 ( pt i=p0 i )s0b i where s0b i = i qb p0 i n P i qb p0 i i=1 (1.1) i=p0 The index can be written, and calculated, in two ways: either as the ratio of two value aggregates, or as an arithmetic weighted average of the price ratios, or price relatives, pt i , for the individual products using the hybrid expenditure shares s0b i as weights.
The expendi- tures are described as hybrid because the prices and quantities belong to two diﬀerent time periods, 0 and b respectively.

The hybrid weights may be obtained by updating the actual expenditure shares in period b, namely pb i , for the price changes occurring between periods b and 0 by multiplying them by the price relatives b and 0, namely p0 i .

Lowe indices are widely used for CPI purposes.


i =Ppb i =pb i qb i qb Laspeyres and Paasche indices 1.21 Any set of quantities could be used in a Lowe index, but there are two special cases which ﬁgure very prominently in the literature and are of considerable importance from a theoretical point of view.

When the quantities are those of the price reference period, that is when b=0, the Laspeyres index is obtained.
When quantities are those of the other period, that is when b=t, the Paasche index is obtained.

It is necessary to consider the properties of Laspeyres and Paasche indi- ces, and also the relationships between them, in more detail.

1.22 The Laspeyres price index, PL, is deﬁned as: PL= n P i=1 n P i=1 pt iq0 i i q0 p0 i n P i=1 ( pt i=p0 i )s0 i (1.2) i denotes the share of the actual expenditure on where s0 commodity i in period 0: that is, p0 i q0 i .
1.23 The Paasche index, PP, is deﬁned as: i =Pp0 i q0 PP= n P i=1 n P i=1 iqt pt i i qt p0 i n P i=1 ( pt i=p0 i )1st i 1 (1.3) iqt that is, pt i=P pt in period t; where st i denotes the actual share of the expenditure iqt on commodity i i.

Notice that the Paasche index is a weighted harmonic average of the price relatives that uses the actual expenditure shares in the later period t as weights.
It follows from equation (1.1) that the Paasche index can also be expressed as a weighted arithmetic ave- rage of the price relatives using hybrid expenditure weights, in which the quantities of t are valued at the prices of 0.

Decomposing current value changes using Laspeyres and Paasche indices 1.24 Laspeyres and Paasche quantity indices are deﬁned in a similar way to the price indices, simply by interchanging the p and q values in formulae (1.2) and (1.3).
They summarize changes over time in the ﬂow of quantities of goods and services consumed.

A Laspeyres quantity index values the quantities at the ﬁxed prices of the earlier period, while the Paasche quantity index uses the prices of the later period.

The ratio of the values of the expenditures in two periods (V ) reﬂects the combined eﬀects of both price and quantity changes.
When Laspeyres and Paasche indices are used, the value change can be exactly decomposed into a price index times a quantity index only if the Laspeyres price (quantity) index is matched with the Paasche quantity (price) index.
Let PLa and QLa denote the Laspeyres price and quantity indices and let PPa and QPa denote the Paasche price and quantity indices: then, PLaQPa:V and PPa QLa:V. 3 CONSUMER PRICE INDEX MANUAL: THEORY AND PRACTICE 1.25 Suppose, for example, a time series of house- hold consumption expenditures at current prices in the national accounts is to be deﬂated by a price index to show changes in real consumption.
To generate a series of consumption expenditures at constant base period prices (whose movements are identical with those of the Laspeyres volume index), the consumption expenditures at current prices must be deﬂated by a series of Paasche price indices.

Ratios of Lowe and Laspeyres indices 1.26 The Lowe index is transitive.

The ratio of two Lowe indices using the same set of qb values is also a Lowe index.
For example, the ratio of the Lowe index for period t+1 with price reference period 0 divided by that for period t also with price reference period 0 is: n P i=1 n P i=1 pt+1 i qb i n .

P i=1 n .

P i=1 iqb pt i i qb p0 i i qb p0 i = n P i=1 n P i=1 pt+1 i qb i iqb pt i =Pt, t+1 Lo (1.4) This is a Lowe index for period t+1 with period t as the price reference period.
This kind of index is, in fact, widely used to measure short-term price movements, such as between t and t+1, even though the quantities may date back to some much earlier period b.

1.27 A Lowe index can also be expressed as the ratio of two Laspeyres indices.

For example, the Lowe index for period t with price reference period 0 is equal to the Laspeyres index for period t with price reference period b divided by the Laspeyres index for period 0 also with price reference period b.
Thus, PLo= n P i=1 n P i=1 iqb pt i i qb p0 i = n P i=1 n P i=1 iqb pt i i qb p0 i n .

P i=1 n .
P i=1 i qb pb i i qb pb i = Pt La P 0 La (1.5) Updated Lowe indices 1.28 It is useful to have a formula that enables a Lowe index to be calculated directly as a chain index, in which the index for period t+1 is obtained by updating the index for period t. Because Lowe indices are tran- sitive, the Lowe index for period t+1 with price refer- ence period 0 can be written as the product of the Lowe index for period t with price reference period 0 multi- plied by the Lowe index for period t+1 with price reference period t. Thus, n P i=1 n P i=1 pt+1 i qb i = i qb p0 i = 2 6 6 4 2 6 6 4 n P i=1 n P i=1 n P i=1 n P i=1 3 7 7 5 3 7 7 5 iqb pt i i qb p0 i iqb pt i i qb p0 i 4 2 6 6 4 n P i=1 n P i=1 pt+1 i qb i iqb pt i 3 7 7 5 n P i=1 pt+1 i pt i stb i (1.6) where the expenditure weights stb deﬁned as: i are hybrid weights i pt stb iqb i n .P i=1 iqb pt i (1.7) i qb i qb i =P pb i by the price relatives pt 1.29 Hybrid weights of the kind deﬁned in equation (1.7) are often described as price-updated weights.
They can be obtained by adjusting the original expenditure weights pb i .

By price-updating the expenditure weights from b to t in this way, the index between t and t+1 can be calculated directly as a weighted average of the price relatives pt+1 i without referring back to the price reference i period 0.
The index can then be linked on to the value of the index in the preceding period t. i=pb =pt Interrelationships between ﬁxed basket indices 1.30 Consider ﬁrst the interrelationship between the Laspeyres and the Paasche indices.

A well-known result in index number theory is that if the price and quantity changes (weighted by values) are negatively correlated, then the Laspeyres index exceeds the Paasche index.
Conversely, if the weighted price and quantity changes are positively correlated, then the Paasche index exceeds the Laspeyres index.

The proof is given in Appendix 15.1 of Chapter 15.

1.31 As consumers are usually price-takers, they typically react to price changes by substituting goods or services that have become relatively cheaper for those that have become relatively dearer.
This is known as the substitution eﬀect, a phenomenon that ﬁgures promi- nently in this manual and the wider literature on index numbers.

Substitution tends to generate a negative correlation between the price and quantity relatives, in which case the Laspeyres index is greater than the Paasche index, the gap between them tending to widen over time.
1.32 In practice, however, statistical oﬃces do not calculate Laspeyres or Paasche indices but instead usually calculate Lowe indices as deﬁned in equation (1.1).

The question then arises of how the Lowe index relates to the Laspeyres and Paasche indices.
It is shown in the text of Chapter 15, and also in Appendix 15.2, that if there are persistent long-term trends in relative prices and if the substitution eﬀect is operative, the Lowe index will tend to exceed the Laspeyres, and therefore also the Fisher and the Paasche indices.
Assuming that period b precedes period 0, the ranking under these conditions will be: Lowe Laspeyres Fisher Paasche Moreover, the amount by which the Lowe exceeds the other three indices will tend to increase, the further back in time period b is in relation to period 0.

1.33 The positioning of period b is crucial.


Given the assumptions about long-term price trends and substitu- tion, a Lowe index will tend to increase as period b is moved backwards in time, or to decrease as period b is moved forwards in time.

While b may have to precede 0 AN INTRODUCTION TO CONSUMER PRICE INDEX METHODOLOGY when the index is ﬁrst published, there is no such restriction on the positioning of b as price and quantity data become available for later periods with passage of time.
Period b can then be moved forwards.

If b is positioned midway between 0 and t, the quantities are likely to be equi-representative of both periods, assuming that there is a fairly smooth transition from the relative quantities of 0 to those of t. In these circumstances, the Lowe index is likely to be close to the Fisher and other superlative indices, and cannot be presumed to have either an upward or a downward bias.

These points are elaborated further below, and also in Chapter 15.
1.34 It is important that statistical oﬃces take these relationships into consideration in deciding upon their policies.

There are obviously practical advantages and ﬁnancial savings from continuing to make repeated use over many years of the same ﬁxed set of quantities to calculate a CPI.
However, the amount by which such a CPI exceeds some conceptually preferred target index, such as a cost of living index (COLI), is likely to get steadily larger the further back in time the period b to which the quantities refer.

Most users are likely to interpret the diﬀerence as upward bias.

A large bias may undermine the credibility and acceptability of the index.
Young index 1.35 Instead of holding constant the quantities of period b, a statistical oﬃce may calculate a CPI as a weighted arithmetic average of the individual price relatives, holding constant the revenue shares of period b.

The resulting index is called a Young index in this manual, again after another index number pioneer.
The Young index is deﬁned as follows: n PYo P i=1 sb i pt i p0 i where sb i i qb pb i n P i qb pb i i=1 (1.8) In the corresponding Lowe index, equation (1.1), the weights are hybrid revenue shares that value the quan- tities of b at the prices of 0.

As already explained, the price reference period 0 is usually later than the weight reference period b because of the time needed to collect and process and revenue data.
In that case, a statistical oﬃce has the choice of assuming that either the quan- tities of period b remain constant or the expenditure shares in period b remain constant.

Both cannot remain constant if prices change between b and 0.

If the expenditure shares actually remained constant between periods b and 0, the quantities must have changed inversely in response to the price changes, which implies an elasticity of substitution of unity.
1.36 Whereas there is a presumption that the Lowe index will tend to exceed the Laspeyres index, it is more diﬃcult to generalize about the relationship between the Young index and the Laspeyres index.

The Young could be greater or less than the Laspeyres depending on how sensitive the quantities are to changes in relative prices.
It is shown in Chapter 15 that with high elasticities of substitution (greater than unity) the Young will tend to exceed the Laspeyres, whereas with low elasticities the Young will tend to be less than the Laspeyres.

1.37 As explained later in this chapter, the Lowe index may be preferred to the Young index because the Young index has some undesirable properties that cause it to fail some critical index number tests (see also Chapter 16).
Geometric Young, Laspeyres and Paasche indices 1.38 In the geometric version of the Young index, a weighted geometric average is taken of the price relatives using the expenditure shares of period b as weights.

It is deﬁned as follows: PGYo n Y i=1 i sb pt i p0 i (1.9) where sb i is deﬁned as above.

The geometric Laspeyres is the special case in which b=0; that is, the expenditure shares are those of the price reference period 0.
Similarly, the geometric Paasche uses the expenditure shares of period t. It should be noted that these geometric indices cannot be expressed as the ratios of value aggregates in which the quantities are ﬁxed.

They are not basket indices and there are no counterpart Lowe indices.
1.39 It is worth recalling that for any set of positive numbers the arithmetic average is greater than, or equal to, the geometric average, which in turn is greater than, or equal to, the harmonic average, the equalities holding only when the numbers are all equal.

In the case of unitary cross-elasticities of demand and constant expen- diture shares, the geometric Laspeyres and Paasche indices coincide.
In this case, the ranking of the indices must be the ordinary Laspeyres the geometric Las- peyres and Paasche the ordinary Paasche, because the indices are, respectively, arithmetic, geometric and har- monic averages of the same price relatives which all use the same set of weights.

1.40 The geometric Young and Laspeyres indices have the same information requirements as their ordin- ary arithmetic counterparts.

They can be produced on a timely basis.


Thus, these geometric indices must be treated as serious practical possibilities for purposes of CPI calculations.

As explained later, the geometric indices are likely to be less subject than their arithmetic counterparts to the kinds of index number biases dis- cussed in later sections.

Their main disadvantage may be that, because they are not ﬁxed basket indices, they are not so easy to explain or justify to users.
Symmetric indices 1.41 A symmetric index is one that makes equal use of the prices and quantities in both the periods com- pared and treats them in a symmetric manner.

There are three particular symmetric indices that are widely used in economic statistics.

It is convenient to introduce them at this point.

As already noted, these three indices are also superlative indices.
1.42 The ﬁrst is the Fisher price index, PF, deﬁned as the geometric average of the Laspeyres and Paasche 5 CONSUMER PRICE INDEX MANUAL: THEORY AND PRACTICE indices; that is, PF p ﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃ PLPP (1.10) 1.43 The second is the Walsh price index, PW.
This is a basket index whose quantities consist of geometric averages of the quantities in the two periods; that is, PW n P i=1 n P i=1 p ﬃﬃﬃﬃﬃﬃﬃﬃﬃqt iq0 i pt i p ﬃﬃﬃﬃﬃﬃﬃﬃﬃqt iq0 i p0 i (1.11) By taking a geometric rather than an arithmetic average of the quantities, equal weight is given to the relative quantities in both periods.

The quantities in the Walsh index can be regarded as being equi-representative of both periods.
1.44 The third index is the To¨rnqvist price index, PT, deﬁned as a geometric average of the price relatives weighted by the average expenditure shares in the two periods.

PT = n Y i=1 ( pt i=p0 i )si (1.12) where si expenditure on product i in the two periods.

is the arithmetic average of the share of si= St i +S0 i 2 (1.13) where the si values are deﬁned as in equations (1.2) and (1.3) above.
1.45 The theoretical attractions of these indices become more apparent in the following sections on the axiomatic and economic approaches to index numbers.

Fixed base versus chain indices 1.46 This topic is examined in Chapter 15.

When a time series of Lowe or Laspeyres indices is calculated using a ﬁxed set of quantities, the quantities become progressively out of date and increasingly irrelevant to the later periods for which prices are being compared.
The base period, in which quantities are set, has to be updated sooner or later and the new index series linked to the old.

Linking is inevitable in the long run.

1.47 In a chain index, each link consists of an index in which each period is compared with the preceding one, the weight and price reference periods being moved forward each period.

Any index number formula can be used for the individual links in a chain index.
For example, it is possible to have a chain index in which the index for t+1 on t is a Lowe index deﬁned as P pt+1qtj=P ptqtj.

The quantities refer to some period that is j periods earlier than the price reference period t. The quantities move forward one period as the price reference period moves forward one period.

If j=0, the chain Lowe becomes a chain Laspeyres, while if j ¼ 1, it becomes a chain Paasche.
1.48 The CPIs in some countries are, in fact, annual chain Lowe indices of this general type, the quantities referring to some year or years that precede the price 6 reference period 0 by a ﬁxed period.

For example, the 12 monthly indices from January 2000 to January 2001, with January 2000 as the price reference period, could be Lowe indices based on price-updated expenditures for 1998.

The 12 indices from January 2001 to January 2002 are then based on price updated expenditures for 1999, and so on.
1.49 The expenditures lag behind the January price reference period by a ﬁxed interval, moving forward a year each January as the price reference period moves forward one year.

Although, for practical reasons, there has to be a time lag between the quantities and the prices when the index is ﬁrst published, it is possible to recal- culate the monthly indices for the current year later, using current expenditure data when they eventually become available.
In this way, it is possible for the long- run index to be an annually chained monthly index, with contemporaneous annual weights.

This method is explained in more detail in Chapter 9.

It is used by one statistical oﬃce.

1.50 A chain index has to be ‘‘path dependent’’.

It must depend on the prices and quantities in all the intervening periods between the ﬁrst and last periods in the index series.

Path dependency can be advantageous or disadvantageous.
When there is a gradual economic transition from the ﬁrst to the last period, with smooth trends in relative prices and quantities, chaining will tend to reduce the index number spread between the Lowe, Laspeyres and Paasche indices, thereby making the movements in the index less dependent on the choice of index number formula.
If 1.51 there are ﬂuctuations in the prices and quantities in the intervening periods, however, chaining may not only increase the index number spread but also distort the measure of the overall change between the ﬁrst and last periods.

For example, suppose all the prices in the last period return to their initial levels in period 0, which implies that they must have ﬂuctuated in between; a chain Laspeyres index will not return to 100.

It will tend to be greater than 100.


If the cycle is repeated with all the prices periodically returning to their original levels, a chain Laspeyres index will tend to drift further and further above 100 even though there may be no long-term upward trend in the prices.

Chaining is therefore not advised when prices ﬂuctuate.

When monthly prices are subject to regular and substantial seasonal ﬂuctuations, for example, monthly chaining cannot be recommended.
Seasonal ﬂuctuations cause serious problems, which are analysed in Chapter 22.

While a number of countries update their expenditure weights annually, the 12-monthly indices within each year are not chain indices but Lowe indices using ﬁxed annual quantities.

1.52 The Divisia index.

If the prices and quantities are continuous functions of time, it is possible to par- tition the change in their total value over time into price and quantity components following the method of Divisia.
As shown in Chapter 15, the Divisia index may be derived mathematically by diﬀerentiating value (i.e.

price multiplied by quantity) with respect to time to obtain two components: a relative-value-weighted price change and a relative-value-weighted quantity change.

AN INTRODUCTION TO CONSUMER PRICE INDEX METHODOLOGY These two components are deﬁned to be price and quantity indices, respectively.

The Divisia is essentially a theoretical index.


In practice, prices can be recorded only at discrete intervals, even if they vary continuously with time.

A chain index may, however, be regarded as a discrete approximation to a Divisia.

The Divisia index itself oﬀers limited practical guidance about the kind of index number formula to choose for the individual links in a chain index.
Axiomatic and stochastic approaches to index numbers 1.53 Various axiomatic approaches to index num- bers are explained in Chapter 16.

These approaches seek to determine the most appropriate functional form for an index by specifying a number of axioms, or tests, that the index ought to satisfy.

They throw light on the properties possessed by diﬀerent kinds of indices, some of which are not intuitively obvious.
Indices that fail to satisfy certain basic or fundamental axioms, or tests, may be rejected completely because they are liable to behave in unacceptable ways.

An axiomatic approach may also be used to rank indices on the basis of their desirable, and undesirable, properties.

First axiomatic approach 1.54 The ﬁrst approach is the traditional test approach pioneered by Irving Fisher.
The price and quantity indices are deﬁned as functions of the two vectors of prices and two vectors of quantities relating to the two periods compared.

The prices and quantities are treated as independent variables, whereas in the eco- nomic approach to index numbers considered later in this chapter the quantities are assumed to be functions of the prices.

1.55 Chapter 16 starts by considering a set of 20 axioms, but only a selection of them is given here by way of illustration.
T1: positivity – the price index and its constituent vectors of prices and quantities should be positive.

T3: identity test – if the price of every product is identical in both periods, then the price index should equal unity, no matter what the quantity vectors are.
T5: proportionality in current prices – if all prices in period t are multiplied by the positive number l, then the new price index should be l times the old price index; i.e., the price index function is (positively) homogeneous of degree one in the components of the period t price vector.

T10: invariance to changes in the units of measure- ment (commensurability test) – the price index does not change if the units in which the products are measured are changed.
T11: time reversal test – if all the data for the two periods are interchanged, then the resulting price index should equal the reciprocal of the original price index.

T14: mean value test for prices – the price index lies between the highest and the lowest price relatives.

T16: Paasche and Laspeyres bounding test – the price index lies between the Laspeyres and Paasche indices.

T17: monotonicity in current prices – if any period t price is increased, then the price index must increase.
1.56 Some of the axioms or tests can be regarded as more important than others.

Indeed, some of the axioms seem so inherently reasonable that it might be assumed that any index number actually in use would satisfy them.

For example, test T10, the commensurability test, says that if the unit of quantity in which a product is measured is changed, say, from a gallon to a litre, the index must be unchanged.
One index that does not satisfy this test is the Dutot index, which is deﬁned as the ratio of the arithmetic means of the prices in the two periods.

As explained later, this is a type of elementary index that is in fact widely used in the early stages of CPI calculation.

1.57 Consider, for example, the average price of salt and pepper.

Suppose it is decided to change the unit of measurement for pepper from grams to ounces while leaving the units in which salt is measured (for example, kilos) unchanged.
As an ounce is equal to 28.35 grams, the absolute value of the price of pepper increases by over 28 times, which eﬀectively increases the weight of pepper in the Dutot index by over 28 times.

1.58 When the products covered by an index are heterogeneous and measured in diﬀerent physical units, the value of any index that does not satisfy the com- mensurability test depends on the purely arbitrary choice of units.

Such an index must be unacceptable con- ceptually.


If the prices refer to a strictly homogeneous set of products that all use the same unit of measurement, the test becomes irrelevant.

1.59 Another important test is T11, the time reversal test.
In principle, it seems reasonable to require that the same result should be obtained whichever of the two periods is chosen as the price reference period: in other words, whether the change is measured forwards in time, i.e., from 0 to t, or backwards in time from t to 0.

The Young index fails this test because an arithmetic average of a set of price relatives is not equal to the reciprocal of the arithmetic average of the reciprocals of the price relatives.
The fact that the conceptually arbitrary deci- sion to measure the change in prices forwards from 0 and t gives a diﬀerent result from measuring backwards from t to 0 is seen by many users as a serious dis- advantage.

The failure of the Young index to satisfy the time reversal test needs to be taken into account by statistical oﬃces.

1.60 Both the Laspeyres and Paasche fail the time reversal test for the same reasons as the Young index.
For example, the formula for a Laspeyres calculated backwards from t to 0, PBL, is: PBL= n P i=1 n P i=1 i qt p0 i iqt pt i 1 PP (1.14) This index is identical to the reciprocal of the (for- wards) Paasche, not to the reciprocal of the (forwards) Laspeyres.

As already noted, the (forwards) Paasche tends to register a smaller increase than the (forwards) Laspeyres so that the Laspeyres index cannot satisfy the 7 CONSUMER PRICE INDEX MANUAL: THEORY AND PRACTICE time reversal test.
The Paasche index also fails the time reversal test.

1.61 In contrast, the Lowe index satisﬁes the time reversal test provided that the quantities qb i remain ﬁxed when the price reference period is changed from 0 to t. The quantities of a Laspeyres index are, however, those of the price reference period by deﬁnition, and must change whenever the price reference period is changed.
The basket for a forwards Laspeyres is diﬀerent from that for a backwards Laspeyres, and the Laspeyres fails the time reversal test in consequence.

1.62 Similarly, the Lowe index is transitive whereas the Laspeyres and Paasche indices are not.

Assuming that a Lowe index uses a ﬁxed set of quantities, qb i , whatever the price reference period, it follows that Lo0, t=Lo0, tkLotk, t where Lo0, t is the Lowe index for period t with period 0 as the price reference period.
The Lowe index that compares t directly with 0 is the same as that calculated indirectly as a chain index through period t k. 1.63 If, on the other hand, the Lowe index is deﬁned in such a way that quantities vary with the price refer- ence period, as in the index P pt+1qtj=P ptqtj con- sidered earlier, the resulting chain index is not transitive.

The chain Laspeyres and chain Paasche indices are special cases of this index.
1.64 In the real world, the quantities do change and the whole point of chaining is to enable the quantities to be continually updated to take account of the changing universe of products.

Achieving transitivity by arbitrarily holding the quantities constant, especially over a very long period of time, does not compensate for the poten- tial biases introduced by using out-of-date quantities.
Ranking of indices using the ﬁrst axiomatic approach 1.65 In Chapter 16 it is shown not only that the Fisher price index satisﬁes all the 20 axioms listed but also, more remarkably, that it is the only possible index that can satisfy all 20 axioms.

Thus, on the basis of this particular set of axioms, the Fisher clearly dominates other indices.

1.66 In contrast to Fisher, the other two symmetric (and superlative) indices deﬁned in equations (1.11) and (1.12) above do not emerge so well from the 20 tests.
In Chapter 16, it is shown that the Walsh price index fails four tests while the To¨ rnqvist index fails nine tests.

Nevertheless, the To¨ rnqvist and the Fisher may be ex- pected to approximate each other quite closely numeri- cally when the data follow relatively smooth trends, as shown in Chapter 19.

1.67 One limitation of the axiomatic approach is that the list of axioms is inevitably somewhat arbitrary.
Some axioms, such as the Paasche and Laspeyres bounding test failed by both To¨ rnqvist and Walsh, could be regarded as dispensable.

Additional axioms or tests can be envisaged, and two further axioms are consid- ered below.

Another problem with a simple application of the axiomatic approach is that it is not suﬃcient to know which tests are failed.

It is also necessary to know 8 how badly an index fails.
Failing badly one major test, such as the commensurability test, might be con- sidered suﬃcient to rule out an index, whereas failing several minor tests marginally may not be very dis- advantageous.

Some further tests 1.68 Consider a further symmetry test.

Reversing the roles of prices and quantities in a price index yields a quantity index of the same functional form as the price index.
The factor reversal test requires that the product of this quantity index and the original price index should be identical with the change in the value of the aggregate in question.

The test is important if, as stated earlier, price and quantity indices are intended to enable changes in the values of aggregates over time to be factored into their price and quantity components in an economically meaningful way.
Another interesting result given in Chapter 16 is that the Fisher index is the only price index to satisfy four minimal tests: T1 (positivity), T11 (time reversal test), T12 (quantity reversal test) and T21 (factor reversal test).
As the factor reversal test implicitly assumes that the prices and quantities must refer either to period 0 or to period t, it is not relevant to a Lowe index in which three periods are involved, b, 0 and t. 1.69 As shown earlier, the product of the Laspeyres price (quantity) index and the Paasche quantity (price) index is identical with the change in the total value of the aggregate in question.
Thus, Laspeyres and Paasche indices may be said to satisfy a weak version of the factor reversal test in that dividing the value change by a Laspeyres (Paasche) price index does lead to a mean- ingful quantity index, i.e., the Paasche (Laspeyres), even though the functional forms of the price and quantity indices are not identical.

1.70 Another test discussed in Chapter 16 is the additivity test.

This is more important from the per- spective of quantity than price indices.
Price indices may be used to deﬂate value changes to obtain implicit quantity changes.

The results may be presented for sub- aggregates such as broad categories of household con- sumption.

Just as expenditure aggregates at current prices are, by deﬁnition, obtained simply by summing individual expenditures, it is reasonable to expect that the changes in the sub-aggregates of a quantity index should add up to the changes in the totals – the addi- tivity test.
Quantity indices such as Laspeyres and Paasche that use a common set of prices to value quantities in both periods must satisfy the additivity test.

Similarly, the Lowe quantity index deﬁned as P p jqt=P p jq0 is also additive.

The Geary–Khamis quantity index (see Annex 4) used to make international comparisons of real consumption and gross domestic product (GDP) between countries is an example of such a Lowe quantity index.
It uses an arithmetically weighted average of the prices in the diﬀerent countries as the common price vector p j to compare the quantities in diﬀerent countries.

1.71 Similarly, an average of the prices in two peri- ods can be used to value the quantities in intertemporal indices.

If the quantity index is also to satisfy the time AN INTRODUCTION TO CONSUMER PRICE INDEX METHODOLOGY reversal test, the average must be symmetrical.
The invariance to proportional changes in current prices test (which corresponds to test T7 listed in Chapter 16, except that the roles of prices and quantities are reversed) requires that a quantity index depend only on the rela- tive, not the absolute, level of the prices in each period.

The Walsh quantity index satisﬁes this test, is additive and satisﬁes the time reversal test as well.

It emerges as a quantity index with some very desirable properties.
1.72 Although the Fisher index itself is not additive, it is possible to decompose the overall percentage change in a Fisher price, or quantity, index into additive com- ponents that reﬂect the percentage change in each price or quantity.

A similar multiplicative decomposition is possible for a To¨ rnqvist price or quantity index.

The stochastic approach and a second axiomatic approach 1.73 Before considering a second axiomatic ap- proach, it is convenient to take the stochastic approach to price indices.
The stochastic approach treats the ob- served price changes or relatives as if they were a random sample drawn from a deﬁned universe whose mean can be interpreted as the general rate of inﬂation.

There can, however, be no single unique rate of inﬂation.

Many possible universes can be deﬁned, depending on which particular sets of expenditures or transactions the user is interested in.

Clearly, the sample mean depends on the choice of universe from which the sample is drawn.
Specifying the universe is similar to specifying the scope of a CPI.

The stochastic approach addresses issues such as the appropriate form of average to take and the most eﬃcient way to estimate it from a sample of price rela- tives, once the universe has been deﬁned.

1.74 The stochastic approach is particularly useful when the universe is reduced to a single type of product.
Because of market imperfections, there may be con- siderable variation in the prices at which the same pro- duct is sold in diﬀerent outlets and also in the price changes observed.

In practice, statistical oﬃces have to estimate the average price change for a single product from a sample of price observations.

Important meth- odological issues are raised, which are discussed in some detail in Chapter 7 and Chapter 20.
The unweighted stochastic approach 1.75 In Chapter 16, the unweighted stochastic approach to index number theory is explained.

If simple random sampling has been used, equal weight may be given to each sampled price relative.

Suppose each price relative can be treated as the sum of two components: a common inﬂation rate and a random disturbance with a zero mean.
Using least squares or maximum likelihood, the best estimate of the common inﬂation rate is the unweighted arithmetic mean of price relatives, an index formula known as the Carli index.

This index is the unweighted version of the Young index and is discussed further below, in the context of elementary price indices.
If the random component is multiplicative, not additive, the best estimate of the common inﬂation rate is given by the unweighted geometric mean of price 1.76 relatives, known as the Jevons index.

The Jevons index may be preferred to the Carli on the grounds that it satisﬁes the time reversal test, whereas the Carli does not.

As explained below, this fact may be decisive when determining the functional form to be used to estimate the elementary indices compiled in the early stages of CPI calculations.
The weighted stochastic approach 1.77 As explained in Chapter 16, a weighted sto- chastic approach can be applied at an aggregative level covering sets of diﬀerent products.

As the products may be of diﬀering economic importance, equal weight should not be given to each type of product.

The pro- ducts may be weighted on the basis of their share in the total value of the expenditures, or other transactions, in some period or periods.
In this case, the index (or its logarithm) is the expected value of a random sample of price relatives (or their logarithms) whose probability of selection is proportional to the expenditure on that type of product in some period, or periods.

Diﬀerent indices are obtained depending on which expenditure weights are used and on whether the price relatives or their logarithms are used.
1.78 Suppose a sample of price relatives is randomly selected with the probability of selection proportional to the expenditure on that type of product in the price reference period 0.

The expected price change is then the Laspeyres price index for the universe.

Other indices may, however, also be obtained using the weighted sto- chastic approach.
Suppose both periods are treated symmetrically and the probabilities of selection are made proportional to the arithmetic mean expenditure shares in both periods 0 and t. When these weights are applied to the logarithms of the price relatives, the expected value of the logarithms is the To¨ rnqvist index, also known as the To¨ rnqvist–Theil index.
From an axiomatic viewpoint, the choice of a symmetric average of the expenditure shares ensures that the time reversal test is satisﬁed, while the choice of the arithmetic mean, as distinct from some other symmetric average, may be justiﬁed on the grounds that the fundamental proportionality in current prices test, T5, is thereby satisﬁed.

1.79 By focusing on price changes, the To¨ rnqvist index emerges as an index with some very desirable prop- erties.
This suggests a second axiomatic approach to indices, in which the focus is shifted from the individual prices and quantities used in the traditional axiomatic approach, to price changes and values shares.

A second axiomatic approach 1.80 A second axiomatic approach is examined in Chapter 16 in which a price index is deﬁned as a func- tion of the two sets of prices, or their ratios, and two sets of values.
Provided the index is invariant to changes in units of measurement, i.e., satisﬁes the commensu- rability test, it makes no diﬀerence whether individual prices or their ratios are speciﬁed.

A set of 17 axioms is postulated which are similar to the 20 axioms considered in the ﬁrst axiomatic approach.
9 CONSUMER PRICE INDEX MANUAL: THEORY AND PRACTICE 1.81 It is shown in Appendix 16.1 that the To¨ rnqvist, or To¨ rnqvist–Theil, is the only price index to satisfy all 17 axioms, just as the Fisher price index is the only index to satisfy all 20 tests in the ﬁrst approach.

However, the To¨ rnqvist index does not satisfy the factor reversal test, so that the implicit quantity index obtained by deﬂating the change in value by the To¨ rnqvist price index is not the To¨ rnqvist quantity index.
The implicit quantity index is therefore not ‘‘best’’ in the sense of satisfying the 17 axioms when these are applied to the quantity, rather than price, indices.

1.82 Zero prices may cause problems for indices based on price ratios, and especially for geometric averages of price ratios.

In particular, if any price tends to zero, one test that may be applied is that the price index ought not to tend to zero or plus inﬁnity.

The To¨ rnqvist does not meet this test.


It is therefore proposed in Chapter 16 that when using the To¨ rnqvist index, care should be taken to bound the prices away from zero in order to avoid a meaningless index number.

1.83 Finally, Chapter 16 examines the axiomatic properties of the Lowe and Young indices.

The Lowe index emerges quite well from the axiomatic approach, satisfying both the time reversal and circularity tests.
On the other hand, the Young index, like the Laspeyres and Paasche indices, fails both tests.

As already explained, however, the attractiveness of the Lowe index depends more on how relevant the ﬁxed quantity weights are to the two periods being compared, that is on the posi- tioning of period b, than its axiomatic properties.

1.84 Although the ‘‘best’’ indices emerging from the two axiomatic approaches, namely Fisher and To¨ rnqvist, are not the same, they have much in common.
As already noted, they are both symmetric indices and they are both superlative indices.

Although their formulae are diﬀerent, they may be expected to behave in similar ways and register similar price movements.

The same type of indi- ces keep emerging as having desirable properties which- ever approach to index theory is adopted, a conclusion that is reinforced by the economic approach to index numbers, which is explained in Chapter 17.
Cost of living index 1.85 Approaching the consumer price index from the standpoint of economic theory has led to the development of the concept of a cost of living index (COLI).

The theory of the COLI was ﬁrst developed by Konus (1924).

It rests on the assumption of optimizing behaviour on the part of a rational consumer.
The COLI for such a consumer has been deﬁned succinctly as the ratio of the minimum expenditures needed to attain the given level of utility, or welfare, under two diﬀerent price regimes.

A more precise deﬁnition and explanation are given in Chapter 17.
1.86 Whereas a Lowe index measures the change in the cost of purchasing a ﬁxed basket of goods and ser- vices resulting from changes in their prices, a COLI measures the change in the minimum cost of maintain- ing a given level of utility, or welfare, that results from changes in the prices of the goods and services consumed.

10 1.87 A COLI is liable to possible misinterpretation because households’ welfare depends on a variety of physical and social factors that have no connection with prices.
Events may occur that impinge directly on wel- fare, such as natural or man-made disasters.

When such events occur, households may need to increase their consumption of goods and services in order to com- pensate for the loss of welfare caused by those events.
Changes in the costs of consumption triggered by events other than changes in prices are irrelevant for a CPI that is not merely intended to measure changes in the prices of consumer goods and services but is generally inter- preted by users as measuring price changes, and only price changes.

In order to qualify as a CPI, a COLI must therefore hold constant not only the consumer’s pre- ferences but all the non-price factors that aﬀect the consumer’s welfare and standard of living.
If a CPI is intended to be a COLI it must be conditional on: – a particular level of utility or welfare; – a particular set of consumer preferences; – a particular state of the physical and social environ- ment.

Of course, Lowe indices are also conditional as they depend on the particular basket of goods and services selected.

1.88 Lowe indices and COLIs have in common the fact that they may both be deﬁned as the ratios of expenditures in two periods.
However, whereas, by deﬁnition, the quantities are ﬁxed in Lowe indices, they vary in response to changes in relative prices in COLIs.

In contrast to the ﬁxed basket approach to index theory, the economic approach explicitly recognizes that the quantities consumed are actually dependent on the pri- ces.

In practice, rational consumers may be expected to adjust the relative quantities they consume in response to changes in relative prices.
A COLI assumes that a consumer seeking to minimize the cost of maintaining a given level of utility will make the necessary adjust- ments.

The baskets of goods and services in the numerator and denominator of a COLI are not there- fore exactly the same.

1.89 The observed expenditure of a rational con- sumer in the selected base period may be assumed to be the minimum expenditure needed to achieve the level of utility enjoyed in that period.
In order to calculate a COLI based on that period, it is necessary to know what would be the minimum expenditure needed to attain precisely the same level of utility if the prices prevailing were those of the second period, other things remaining equal.

The quantities purchased under these assumed conditions are likely to be hypothetical.

They will not be the quantities actually consumed in the second period if other factors, including the resources available to the consumer, have changed.
1.90 The quantities required for the calculation of the COLI in at least one of the periods are not likely to be observable in practice.

The COLI is not an operational index that can be calculated directly.

The challenge is therefore to see whether it is possible to ﬁnd methods of estimating a COLI indirectly or at least to ﬁnd upper and lower bounds for the index.
There is also AN INTRODUCTION TO CONSUMER PRICE INDEX METHODOLOGY considerable interest in establishing the relationship between a COLI and Lowe indices, including Laspeyres and Paasche, that can be calculated.
Upper and lower bounds on a cost of living index 1.91 It follows from the deﬁnition of a Laspeyres index that, if the consumer’s income were to change by the same proportion as the change in the Laspeyres index, the consumer must have the possibility of pur- chasing the same basket of products as in the base period.

The consumer cannot be worse oﬀ.

However, if relative prices have changed, a utility-maximizing con- sumer would not continue to purchase the same quan- tities as before.
The consumer would be able to achieve a higher level of utility by substituting, at least marginally, products that have become relatively cheaper for those that have become dearer.

As a COLI measures the change in the minimum expenditures needed to maintain a constant level of utility, the COLI based on the ﬁrst period will increase by less than the Laspeyres index.
1.92 By a similar line of reasoning, it follows that when relative prices change, the COLI based on the second period must increase by more than the Paasche index.

As explained in more detail in Chapter 17, the Laspeyres index provides an upper bound to the COLI based on the ﬁrst period and the Paasche a lower bound to the COLI based on the second period.

It should be noted that there are two diﬀerent COLIs involved here: one based on the ﬁrst period and the other based on the second period.
In general, however, the two COLIs are unlikely to diﬀer much.

1.93 Suppose that the theoretical target index is a COLI, but that, for practical reasons, the CPI is actually calculated as a Lowe index in which the quantities refer to some period b that precedes the price reference period 0.
One important conclusion to be drawn from this pre- liminary analysis is that as the Lowe may be expected to exceed the Laspeyres, assuming long-term price trends and substitution, while the Laspeyres may in turn be expected to exceed the COLI, the widely used Lowe index may be expected to have an upward bias.

This point has had a profound inﬂuence on attitudes towards CPIs in some countries.
The bias results from the fact that, by deﬁnition, ﬁxed basket indices, including Laspeyres, do not permit any substitution between products in response to changes in relative prices.

It is therefore usually de- scribed as ‘‘substitution bias’’.

A Paasche index would be expected to have a downward substitution bias.

Some special cases 1.94 The next step is to establish whether there are special conditions under which it may be possible to measure a COLI exactly.
In Chapter 17 it is shown that if the consumer’s preferences are homothetic – that is, each indiﬀerence curve has the same shape, each being a uniform enlargement, or contraction, of each other – then the COLI is independent of the utility level on which it is based.

The Laspeyres and Paasche indices provide upper and lower bounds to the same COLI.
1.95 One interesting special case occurs when the preferences can be represented by the so-called ‘‘Cobb– Douglas’’ function in which the cross-elasticities of demand between the various products are all unity.

Con- sumers adjust the relative quantities they consume inver- sely in proportion to the changes in relative prices so that expenditure shares remain constant.

With Cobb–Douglas preferences, the geometric Laspeyres provides an exact measure of the COLI.
As the expenditure shares remain constant over time, all three geometric indices – Young, Laspeyres and Paasche – coincide with each other and with the COLI.

Of course, the arithmetic versions of these indices do not coincide in these circumstances, because the baskets in periods b, 0 and t are all diﬀerent as substitu- tions take place in response to changes in relative prices.
1.96 One of the more famous results in index num- ber theory is that if the preferences can be represented by a homogeneous quadratic utility function, the Fisher index provides an exact measure of the COLI (see Chapter 17).
Even though consumers’ preferences are unlikely to conform exactly with this particular func- tional form, this result does suggest that, in general, the Fisher index is likely to provide a close approximation to the underlying unknown COLI and certainly a much closer approximation than either the arithmetic Las- peyres or Paasche indices.

Estimating COLIs by superlative indices 1.97 The intuition – that the Fisher index approx- imates the COLI – is corroborated by the following line of reasoning.
Diewert (1976) noted that a homogeneous quadratic is a ﬂexible functional form that can provide a second-order approximation to other twice-diﬀerentiable functions around the same point.

He then described an index number formula as superlative when it is exactly equal to the COLI based on a certain functional form and when that functional form is ﬂexible, e.g., a homogeneous quadratic.

The derivation of these results, and further explanation, is given in detail in Chapter 17.
In contrast to the COLI based on the true but unknown utility function, a superlative index is an actual index number that can be calculated.

The practical signiﬁcance of these results is that they provide a theoretical justiﬁcation for expecting a superlative index to provide a fairly close approximation to the underlying COLI in a wide range of circumstances.

1.98 Superlative indices as symmetric indices.

The Fisher is by no means the only example of a superlative index.
In fact, there is a whole family of superlative indices.

It is shown in Chapter 17 that any quadratic mean of order r is a superlative index for each value of r=0.

A quadratic mean of order r price index P r is deﬁned as follows: Pr v u u r t r=2 ﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃ !

pt n i P p0 i=1 i s0 i r=2 v u u r t ﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃ !

p0 n i P pt i=1 i st i (1.15) where s0 above.


i and st i are deﬁned as in equations (1.2) and (1.3) 11 CONSUMER PRICE INDEX MANUAL: THEORY AND PRACTICE 1.99 The symmetry of the numerator and denomi- nator of equation (1.15) should be noted.
A distinctive feature of equation (1.15) is that it treats the price changes and expenditure shares in both periods sym- metrically, whatever value is assigned to the parameter r. Three special cases are of interest: – when r=2, equation (1.1) reduces to the Fisher price index; – when r=1 it is equivalent to the Walsh price index; – in the limit as r !

0, it equals the To¨ rnqvist index.


These indices were introduced earlier as examples of indices that treat the information available in both peri- ods symmetrically.

Each was originally proposed long before the concept of a superlative index was developed.

1.100 Choice of superlative index.

Chapter 17 ad- dresses the question of which superlative formula to choose in practice.
As each may be expected to approx- imate to the same underlying COLI, it may be inferred that they ought also to approximate to each other.

The fact that they are all symmetric indices reinforces this conclusion.

These conjectures tend to be borne out in practice by numerical calculations.

So long as the pa- rameter r does not lie far outside the range 0 to 2, superlative indices tend to be very close to each other.
In principle, however, there is no limit on r and it has recently been shown that as r becomes larger, the formula tends to assign increasing weight to the extreme price relatives and the resulting superlative indices may diverge signiﬁcantly from each other.

Only when the absolute value of r is small, as in the case of the three commonly used superlative indices (Fisher, Walsh and To¨ rnqvist), is the choice of superlative index unimportant.
1.101 Both the Fisher and the Walsh indices date back nearly a century.

The Fisher index owes its pop- ularity to the axiomatic, or test, approach, which Fisher himself was instrumental in developing.

As already noted, it dominates other indices using the ﬁrst axiom- atic approach, while the To¨ rnqvist dominates using the second axiomatic approach outlined above.
The fact that the Fisher and the To¨ rnqvist are both superla- tive indices whose use can be justiﬁed on economic grounds suggests that, from a theoretical point of view, it may not be possible to improve on them for CPI purposes.

Representativity bias 1.102 The fact that the Walsh index is a Lowe index that is also superlative suggests that the bias in other Lowe indices depends on the extent to which their quantities deviate from those in the Walsh basket.

This can be viewed from another angle.


1.103 As the quantities in the Walsh basket are geometric averages of the quantities in the two periods, equal importance is assigned to the relative, as distinct from the absolute, quantities in both periods.

The Walsh basket may therefore be regarded as being the basket that is most representative of both periods.
If equal importance is attached to the consumption patterns in the two periods, the optimal basket for a Lowe index ought to be the most representative basket.

The Walsh 12 index then becomes the conceptually preferred target index for a Lowe index.
1.104 Suppose that period b, for which the quan- tities are actually used in the Lowe index, lies midway between 0 and t. In this case, assuming fairly smooth trends in the relative quantities, the actual basket in period b is likely to approximate to the most repre- sentative basket.

Conversely, the further away that period b is from the midpoint between 0 and t, the more the relative quantities of b are likely to diverge from those in the most representative basket.
In this case, the Lowe index between periods 0 and t that uses period b quantities is likely to exceed the Lowe index that uses the most representative quantities by an amount that becomes progressively larger the further back in time period b is positioned.

The excess constitutes ‘‘bias’’ if the latter index is the target index.
The bias can be attributed to the fact that the period b quantities tend to become increasingly unrepresentative of a comparison between 0 and t the further back period b is positioned.

The underlying economic factors responsible are, of course, exactly the same as those that give rise to bias when the target index is the COLI.

Thus, certain kinds of indices can be regarded as biased without invoking the concept of a COLI.
Conversely, the same kinds of indices tend to emerge as being preferred, whether or not the objective is to estimate a cost of living bias.

If 1.105 interest is focused on short-term price movements, the target index is an index between con- secutive time periods t and t+1.

In this case, the most representative basket has to move forward one period as the index moves forward.

Choosing the most repre- sentative basket implies chaining.
Similarly, chaining is also implied if the target index is a COLI between t and t+1.

In practice, the universe of products is continually changing as well.

As the most representative basket moves forward, it is possible to update the set of products covered, as well as take account of changes in the relative quantities of products that were covered previously.
1.106 As superlative indices Data requirements and calculation issues require price and expenditure data for both periods, and as expenditure data are usually not available for the current period, it is not feasible to calculate a superlative CPI, at least at the time that a CPI is ﬁrst published.

In practice, CPIs tend to be Lowe indices with ﬁxed quantities or annually updated chain Lowe indices.
In the course of time, however, the requisite expenditure data may become available, enabling a superlative CPI to be calculated subsequently.

Users will ﬁnd it helpful for superlative CPIs to be published retrospectively as they make it possible to evaluate the properties and behaviour of the oﬃcial index.

Superlative CPIs can be treated as sup- than plementary indices replace, the original indices, if the policy is not to revise the oﬃcial index.
complement, rather that 1.107 Chapter 17 notes that, in practice, CPIs are usually calculated in stages (see also Chapters 9 and 20) and addresses the question of whether indices calculated this way are consistent in aggregation: that is, have the AN INTRODUCTION TO CONSUMER PRICE INDEX METHODOLOGY same values whether calculated in a single operation or in two stages.

The Laspeyres index is shown to be exactly consistent, but the superlative indices are not.
The widely used Fisher and To¨ rnqvist indices are nevertheless shown to be approximately consistent.

Allowing for substitution 1.108 Chapter 17 examines one further index pro- posed recently, the Lloyd–Moulton index, PLM, deﬁned as follows: ( n PLM P i=1 ) 1 1s s0 i 1s pt i p0 i s 6¼ 1 (1.16) The parameter s, which must be non-negative, is the elasticity of substitution between the products covered.
It reﬂects the extent to which, on average, the various products are believed to be substitutes for each other.

The advantage of this index is that it may be expected to be free of substitution bias to a reasonable degree of approximation, while requiring no more data than a Lowe or Laspeyres index.
It is therefore a practical possibility for CPI calculation, even for the most recent periods, although it is likely to be diﬃcult to obtain a satisfactory, acceptable estimate of the numerical value of the elasticity of substitution, the parameter used in the formula.

Aggregation issues 1.109 It has been assumed up to this point that the COLI is based on the preferences of a single repre- sentative consumer.
Chapter 18 examines the extent to which the various conclusions reached above remain valid for CPIs that are actually compiled for groups of households.

The general conclusion is that essentially the same relationships hold at an aggregate level, although some additional issues arise which may require addi- tional assumptions.

1.110 One issue is how to weight individual house- holds.
Aggregate indices that weight households by their expenditures are called ‘‘plutocratic’’, while those that assign the same weight to each household are called ‘‘democratic’’.

Another question is whether, at any one point of time, there is a single set of prices or whether diﬀerent households face diﬀerent prices.
In general, when deﬁning the aggregate indices it is not necessary to assume that all households are confronted by the same set of prices, although the analysis is naturally simpliﬁed if there is only a single set.

1.111 A plutocratic aggregate COLI assumes that each individual household minimizes the cost of attain- ing a given level of utility when confronted by two dif- ferent sets of prices, the aggregate COLI being deﬁned as the ratio of the aggregate minimum costs over all house- holds.
As in the case of a single household, it is recog- nized that the aggregate COLI that is appropriate for CPI purposes must be conditional on the state of a par- ticular set of environmental variables, typically those in one or other of the periods compared.

The environment must be understood in a broad sense to refer not only to the physical environment but also to the social and po- litical environment.
1.112 Like the index for a single representative con- sumer, an aggregate COLI cannot be calculated directly, but it may be possible to calculate aggregate Laspeyres and Paasche indices that bound their respective COLIs from above or below.

If there is only one single set of na- tional prices, the aggregate plutocratic Laspeyres index reduces to an ordinary aggregate Laspeyres index.
As the aggregate plutocratic Laspeyres and Paasche can, in principle, be calculated, so can the aggregate plutocratic Fisher index.

It is argued in Chapter 18 that this should normally provide a good approximation to the aggregate plutocratic COLI.
1.113 Chapter 18 ﬁnally concludes that, in principle, both democratic and plutocratic Laspeyres, Paasche and Fisher indices could be constructed by a statistical agency, provided that information on household-speciﬁc price relatives and expenditures is available for both periods.

If expenditure information is available only for the ﬁrst period, then only the Laspeyres democratic and plutocratic indices can be constructed.

The data require- ments are rather formidable, however.
The required data are unlikely to be available for individual households in practice and, even if they were to be, they could be sub- ject to large errors.

Illustrative numerical data 1.114 Chapter 19 presents some numerical examples using an artiﬁcial data set.

The purpose is not to illus- trate the methods of calculation as such, but rather to demonstrate how diﬀerent index number formulae can yield very diﬀerent numerical results.
Hypothetical but economically plausible prices, quantities and expendi- tures are given for six commodities over ﬁve periods of time.

In general, diﬀerences between the diﬀerent for- mulae tend to increase with the variance of the price relatives.

They also depend on the extent to which the prices follow smooth trends or ﬂuctuate.

1.115 The numerical results are striking.

For exam- ple, the Laspeyres index over the ﬁve periods registers an increase of 44 per cent while the Paasche falls by 20 per cent.
The two commonly used superlative indices, To¨ rnqvist and Fisher, register increases of 25 per cent and 19 per cent respectively, an index number spread of only 6 points compared with the 64-point gap between the Laspeyres and Paasche.

When the indices are chained, the chain Laspeyres and Paasche indices reg- ister increases of 33 per cent and 12 per cent respectively, reducing the gap between the two indices from 64 to 21 points.
The chained To¨ rnqvist and Fisher indices regis- ter increases of 22.26 per cent and 22.24 per cent respectively, being virtually identical numerically.

These results show that the choice of index formula and method does matter.

Seasonal products 1.116 As explained in Chapter 22, the existence of seasonal products poses some intractable problems and 13 CONSUMER PRICE INDEX MANUAL: THEORY AND PRACTICE serious challenges for CPI compilers and users.
Seasonal products are products that are either: However, this remains an area in which more research needs to be done.

– not available during certain seasons of the year; or – are available throughout, but their prices or quantities are subject to regular ﬂuctuations that are synchro- nized with the season or time of the year.

There are two main sources of seasonal ﬂuctuations: the climate and custom.
Month-to-month movements in a CPI may sometimes be so dominated by seasonal inﬂu- ences that it is diﬃcult to discern the underlying trends in prices.

Conventional seasonal adjustment programmes may be applied, but these may not always be satisfactory.
The problem is not conﬁned to interpreting movements in the CPI, as seasonality creates serious problems for the compilation of a CPI when some of the products in the basket regularly disappear and reappear, thereby break- ing the continuity of the price series from which the CPI is built up.

There is no panacea for seasonality.

A con- sensus on what is best practice in this area has not yet been formed.
Chapter 22 examines a number of diﬀerent ways in which the problems may be tackled using an artiﬁcial data set to illustrate the consequences of using diﬀerent methods.

1.117 One possibility is to exclude seasonal products from the index, but this may be an unacceptable reduc- tion in the scope of the index, as seasonal products can account for a signiﬁcant proportion of total household consumption.
Assuming seasonal products are retained, one solution is to switch the focus from month-to-month movements in the index to changes between the same month in successive years.

In some countries, it is com- mon for the media and other users, such as central banks, to focus on the annual rate of inﬂation between the most recent month and the same month in the previous year.
This year-on-year ﬁgure is much easier to interpret than month-to-month changes, which can be somewhat volatile, even in the absence of seasonal ﬂuctuations.

1.118 This approach is extended in Chapter 22 to the concept of a rolling year-on-year index that com- pares the prices for the most recent 12 months with the corresponding months in the price reference year.

The resulting rolling year indices can be regarded as sea- sonally adjusted price indices.
They are shown to work well using the artiﬁcial data set.

Such an index can be regarded as a measure of inﬂation for a year that is centred around a month that is six months earlier than the last month in the rolling index.
For some purposes, this time lag may be disadvantageous, but in Chapter 22 it is shown that under certain conditions the current month year-on-year monthly index, to- gether with the previous month’s year-on-year monthly index, can successfully predict the rolling year index that is centred on the current month.
Of course, rolling year indices and similar analytic constructs are not intended to replace the monthly or quarterly CPI but to provide supplementary information that can be ex- tremely useful to users.

They can be published alongside the oﬃcial CPI.

1.119 Various methods of dealing with the breaks in price series caused by the disappearance and reappear- ance of seasonal products are examined in Chapter 22.
14 Elementary price indices 1.120 As explained in Chapters 9 and 20, the calcu- lation of a CPI proceeds in stages.

In the ﬁrst stage, ele- mentary price indices are estimated for the elementary expenditure aggregates of a CPI.

In the second stage, these elementary indices are aggregated, or averaged, to obtain higher-level indices using the elementary expenditure aggregates as weights.
An elementary aggregate consists of the expenditures on a small and relatively homoge- neous set of products deﬁned within the consumption classiﬁcation used in the CPI.

As explained in Chapter 6, statistical oﬃces usually select a set of representative pro- ducts within each aggregate and then collect samples of their prices from a number of diﬀerent outlets.

The ele- mentary aggregates serve as strata for sampling purposes.
1.121 The prices collected at the ﬁrst stage are typi- cally not prices observed in actual transactions between diﬀerent economic units, but the prices at which the products are oﬀered for sale in retail outlets of one kind or another.

In principle, however, a CPI measures changes in the prices paid by households.

These prices may actually vary during the course of a month, which is typically the time period to which the CPI relates.
In principle, therefore, the ﬁrst step should be to average the prices at which some product is sold during the period, bearing in mind that the price may vary even for the same product sold in the same outlet.

In general, this is not a practical possibility.


However, when the outlet is an electronic point of sale at which all the individual pro- ducts are ‘‘scanned’’ as they are sold, the values of the transactions are actually recorded, thereby making it possible to calculate an average price instead of simply recording the oﬀer price at a single point of time.

Some use of scanner data is already made for CPI purposes and it may be expected to increase over the course of time.
1.122 Once the prices are collected for the repre- sentative products in a sample of outlets, the question arises of what is the most appropriate formula to use to estimate an elementary price index.

This topic is con- sidered in Chapter 20.

It was comparatively neglected until a number of papers in the 1990s provided much clearer insights into the properties of elementary in- dices and their relative strengths and weaknesses.
The quality of a CPI depends heavily on the quality of the elementary indices which are the building blocks from which CPIs are constructed.

1.123 Prices are collected for the same product in the same outlet over a succession of time periods.

An ele- mentary price index is therefore typically calculated from two sets of matched price observations.
Here it is assumed that there are no missing observations and no changes in the quality of the products sampled so that the two sets of prices are perfectly matched.

The treat- ment of new and disappearing products, and of quality change, is a separate and complex issue in its own right.

It is outlined below, and discussed in detail in Chapters 7, 8 and 21.
AN INTRODUCTION TO CONSUMER PRICE INDEX METHODOLOGY Weights within elementary aggregates 1.124 In most cases, the price indices for elementary aggregates are calculated without the use of explicit ex- penditure weights.

Whenever possible, however, weights should be used that reﬂect the relative importance of the sampled items, even if the weights are only approximate.

In many cases, the elementary aggregate is simply the lowest level at which any reliable weighting information is available.
In this case, the elementary index has to be calculated without the use of weights.

Even in this case, however, it should be noted that when the items are selected with probabilities proportional to the size of some relevant variable such as sales, for example, weights are implicitly introduced by the sampling selec- tion procedure.
1.125 For certain elementary aggregates, informa- tion about sales of particular items, market shares and regional weights may be used as explicit weights within an elementary aggregate.

Weights within elementary aggregates may be updated independently, and possibly more often than the elementary aggregates themselves (which serve as weights for the higher-level indices).

1.126 For example, assume that the number of suppliers of a certain product, such as petrol, is limited.
The market shares of the suppliers may be known from business survey statistics and can be used as weights in the calculation of an elementary aggregate price index for petrol.

As another example, prices for water may be collected from a number of local water supply services where the population in each local region is known.
The relative size of the population in each region may then be used as a proxy for the relative consumption expen- ditures to weight the price in each region to obtain the elementary aggregate price index for water.

Interrelationships between different elementary index formulae 1.127 Useful insights into the properties of various formulae that have been used, or considered, for ele- mentary price indices may be gained by examining the mathematical interrelationships between them.
Chapter 20 provides a detailed analysis of such relationships.

As it is assumed that there are no explicit weights avail- able, the various formulae considered all make use of unweighted averages: that is, simple averages in which the various items are equally weighted.

There are two basic options for an elementary index: – some kind of simple average of the price ratios or relatives; – the ratio of some kind of simple average of the prices in the two periods.
In the case of a geometric average, the two methods coincide, as the geometric average of the price ratios or relatives is identical to the ratio of the geometric average prices.
1.128 Using the ﬁrst of the above options, three possible elementary price indices are: – a simple arithmetic average of the price relatives, is the known as the Carli unweighted version of the Young index; index, or PC; the Carli – a simple geometric average of the price relatives, known as the Jevons index, or PJ; the Jevons is the unweighted version of the geometric Young index; – a simple harmonic average of the price relatives, or PH.
As noted earlier, for any set of positive numbers the arithmetic average is greater than, or equal to, the geometric average, which in turn is greater than, or equal to, the harmonic average, the equalities holding only when the numbers are all equal.

It follows that PC PJ PH.

1.129 It is shown in Chapter 20 that the gaps between the three indices widen as the variance of the price relatives increases.
The choice of formula becomes more important the greater the diversity of the price movements.

PJ can be expected to lie approximately halfway between PC and PH.

1.130 Using the second of the options, three possible indices are: – the ratio of the simple arithmetic average prices, known as the Dutot index, or PD; – the ratio of the simple geometric averages, again the Jevons index, or PJ; – the ratio of the simple harmonic averages, or PH.
The ranking of ratios of diﬀerent kinds of average are not predictable.

For example, the Dutot, PD, could be greater or less than the Jevons, PJ.

1.131 The Dutot can also be expressed as a weighted average of the price relatives in which the prices of period 0 serve as the weights: PD .

n pt i .
n pt i .

n p0 i n P i=1 n P i=1 = n P i=1 p0 i pt i p0 i n P i=1 p0 i (1.17) As compared with the Carli, which is a simple average of the price relatives, the Dutot gives more weight to the price relatives for the products with high prices in per- iod 0.

It is nevertheless diﬃcult to provide an economic rationale for this kind of weighting.

Prices are not expenditures.


If the products are homogeneous, very few quantities are likely to be purchased at high prices if the same products can be purchased at low prices.

If the products are heterogeneous, the Dutot should not be used anyway, as the quantities are not commensurate and not additive.

is useful 1.132 While it to establish the inter- relationships between the various indices, they do not actually help decide which index to choose.
However, as the diﬀerences between the various formulae tend to increase with the dispersion of the price relatives, it is clearly desirable to deﬁne the elementary aggregates in such a way as to try to minimize the variation in the price movements within each aggregate.

The less variation there is, the less diﬀerence the choice of index formula makes.
As the elementary aggregates also serve as strata for sampling purposes, minimizing the variance in the price relatives within the strata will also reduce the sampling error.

15 CONSUMER PRICE INDEX MANUAL: THEORY AND PRACTICE Axiomatic approach to elementary indices 1.133 One way to decide between the various ele- mentary indices is to exploit the axiomatic approach outlined earlier.

A number of tests are applied to the elementary indices in Chapter 20.
1.134 The Jevons index, PJ, satisﬁes all the selected tests.

It dominates the other indices in the way that the Fisher tends to dominate other indices at an aggregative level.

The Dutot index, PD, fails only one, the com- mensurability test.

This failure is critical, however.

It reﬂects the fundamental point made earlier that when the quantities are not additive from an economic view- point, the prices are also not additive and hence cannot be meaningfully averaged.
However, PD performs well when the sampled products are homogeneous.

The key issue for the Dutot is therefore how heterogeneous are the products within the elementary aggregate.

If the products are not suﬃciently homogeneous for their quantities to be additive, the Dutot should not be used.

1.135 Although the Carli index, PC, has been widely used in practice, the axiomatic approach shows it to have some undesirable properties.
In particular, as the unweighted version of the Young index, it fails the time reversal and transitivity tests.

This is a serious disadvan- tage, especially as elementary indices are often monthly chain indices.

A consensus has emerged that the Carli may be unsuitable because it is liable to have a sig- niﬁcant upward bias.

This is illustrated by numerical example in Chapter 9.

Its use is not sanctioned for the Harmonized Indices of Consumer Prices used within the European Union.
Conversely, the harmonic average of the price relatives, PH, is liable to have an equally sig- niﬁcant downward bias; anyway, it does not seem to be used in practice.

1.136 Based on the axiomatic approach, the Jevons emerges as the preferred index, but its use may not be appropriate in all circumstances.

If one observation is zero, the geometric mean is zero.
The Jevons is sensitive to extreme falls in prices; it may be necessary to impose upper and lower bounds on the individual price relatives when using the Jevons.

Economic approach to elementary indices 1.137 The economic approach to elementary indices is explained in Chapter 20.

The sampled products for which prices are collected are treated as if they con- stituted a basket of goods and services purchased by rational utility-maximizing consumers.
The objective is then to estimate a conditional cost of living index cov- ering the set of products in question.

1.138 It should be noted, however, that the diﬀer- ences between the prices of the sampled products do not necessarily mean that the products are qualitatively diﬀerent.

If markets were perfect, relative prices should reﬂect relative costs of production and relative utilities.

In fact, price diﬀerences may occur simply because of market imperfections.
For example, exactly the same products may be bought and sold at diﬀerent prices in diﬀerent outlets simply because consumers lack infor- 16 mation about the prices charged in other outlets.

Pro- ducers may also practise price discrimination, charging diﬀerent customers diﬀerent prices for exactly the same products.

Price discrimination is widespread in many service industries.
When the price diﬀerences are a result of market imperfections, consumers cannot be expected to react to changes in the relative prices of products in the same way as they would if they were well informed and had free choice.

1.139 In any case, assuming there is no information about quantities or expenditures within an elementary aggregate, it is not possible to calculate any kind of superlative index.
So the conditional cost of living index at the level of an elementary aggregate can be estimated only on the assumption that certain special conditions apply.

1.140 There are two special cases of some interest.

The ﬁrst case is where the underlying preferences are so- called Leontief preferences.

With these preferences relative quantities remain ﬁxed whatever the relative prices.

No substitutions are made in response to changes in relative prices.

The cross-elasticities of demand are zero.


With Leontief preferences, a Laspeyres index pro- vides an exact measure of the cost of living index.

In this case, the Carli calculated for a random sample would provide an estimate of the cost of living index provided that the items were selected with probabilities propor- tional to the population expenditure shares.
It might appear that if the items were selected with probabilities proportional to the population quantity shares, the sample Dutot would provide an estimate of the popu- lation Laspeyres.

However, assuming that the basket for the Laspeyres index contains a number of heterogeneous products whose quantities are not additive, the quantity shares, and hence the probabilities, are undeﬁned.
1.141 The second case is one already considered above, namely when the preferences can be represented by a Cobb–Douglas function.

As already explained, with these preferences, the geometric Laspeyres would pro- vide an exact measure of the cost of living index.

In this case, the Carli calculated for a random sample would provide an unbiased estimate of the cost of living index, provided that the items were selected with probabilities proportional to the population expenditure shares.
1.142 On the economic approach, the choice between the sample Jevons and the sample Carli rests on which is likely to approximate the more closely to the underlying COLI: in other words, on whether the demand cross- elasticities are likely to be closer to unity or zero, on average.

In practice, the cross-elasticities could take on any value ranging up to plus inﬁnity for an elementary aggregate in which the sampled products were strictly homogeneous, i.e., perfect substitutes.
It should be noted that in the limiting case in which the sampled products are homogeneous, there is only a single kind of product and therefore no index number problem: the price index is given by the ratio of the unit values in the two periods.

It may be conjectured that, on average, the cross-elasti- cities are likely to be closer to unity than zero for most elementary aggregates so that, in general, the Jevons index is likely to provide a closer approximation to the cost of living index than the Carli.
In this case, the Carli must be viewed as having an upward bias.

AN INTRODUCTION TO CONSUMER PRICE INDEX METHODOLOGY 1.143 It is worth noting that the use of the Jevons index does not imply, or assume, that expenditure shares remain constant.

Obviously, a geometric average of the price relatives can be calculated whatever changes do or do not occur in the expenditure shares, in practice.
What the economic approach shows is that if the expenditure shares remain constant (or roughly constant), then the Jevons can be expected to provide a good estimate of the underlying cost of living index.
The insight provided by the economic approach is that the Jevons is likely to provide a closer approximation to the cost of living index than the Carli because a signiﬁcant amount of substitution is more likely than no substitution, espe- cially as elementary aggregates should be deliberately constructed in such a way as to group together similar items that are close substitutes for each other.
1.144 An alternative to the Jevons, PJ, would be a geometric average of PC and PH, an index labelled PCSWD in Chapter 20.

This could be justiﬁed on grounds treating the data in both periods symmetrically of without invoking any particular assumption about the form of the underlying preferences.

It is also shown in Chapter 20 that the geometric average of PC and PH is likely to be very close to PJ, so that the latter may be preferred on the grounds that it is a simpler concept and easier to compile.
1.145 It may be concluded that, based on the eco- nomic approach, as well as the axiomatic approach, the Jevons emerges as the preferred index in general, although there may be cases in which little or no sub- stitution takes place within the elementary aggregate and the Carli might be preferred.

The index compiler must make a judgement on the basis of the nature of the products actually included in the elementary aggregate.
1.146 The above discussion has also thrown light on some of the sampling properties of the elementary indices.

If the products in the sample are selected with probabilities proportional to expenditures in the price reference period: – the sample (unweighted) Carli index provides an unbiased estimate of the population Laspeyres; – the sample (unweighted) Jevons index provides an unbiased estimate of the population geometric Las- peyres.
These results hold irrespective of what the underlying cost of living index may be.

Concepts, scope and classiﬁcations 1.147 The purpose of Chapter 3 of the manual is to deﬁne and clarify a number of basic concepts underlying a CPI and to explain the scope of the index: that is, the set of goods and services and the set of households that the index is intended to cover, in principle.

Chapter 3 also examines the structure of the classiﬁcation of con- sumer goods and services used.
1.148 While the general purpose of a CPI is to measure changes in the prices of consumption goods and services, there are a number of concepts that need to be deﬁned precisely before an operational deﬁnition of a CPI can be arrived at.

The concept of consumption is an imprecise one that can be interpreted in several diﬀerent ways, each of which may lead to a diﬀerent CPI.
It is also necessary to decide whether the index is meant to cover all consumers, i.e., all households, or just a par- ticular group of households.

The scope of a CPI is inevitably inﬂuenced by what is intended, or believed, to be the main use of the index.

Compilers also need to remember that the index may be used as proxy for a general price index and used for purposes other than those for which it is intended.
1.149 The word ‘‘consumer’’ can be used to refer both to a type of economic unit and to a type of pro- duct.

To avoid confusion here, the term consumption good or service will be used where necessary, rather than consumer good or service.

A consumption good or ser- vice provides utility to its user.

It may be deﬁned as a good or service that members of households use, directly or indirectly, to satisfy their own personal needs and wants.

‘‘Utility’’ should be interpreted in a broad sense.
It is simply the generic, technical term preferred by economists for the beneﬁt or welfare that individuals or households derive from the use of a consumer good or service.

1.150 A CPI is generally understood to be a price index that measures changes in the prices of consump- tion goods and services acquired and used by house- holds.
In principle, more broadly based price indices can be deﬁned whose scope extends beyond consumption goods and services to include the prices of physical assets such as land or dwellings.

Such indices may be useful as broad measures of inﬂation as perceived by households, but most CPIs are conﬁned to consumption goods and services.

These may include the prices of the ﬂows of services provided by assets such as dwellings, even though the assets themselves may be excluded.
In any case, the prices of ﬁnancial assets such as bonds, shares or other marketable securities purchased by house- holds are generally regarded as being outside the scope of a CPI.

Acquisitions and uses 1.151 The times at which households acquire and use consumption goods or services are generally not the same.

Goods are typically acquired at one point in time and used at some other point in time, or even used repeatedly over an extended period of time.
The time of acquisition of a good is the moment at which the legal or eﬀective economic ownership of the good passes to the consumer.

In a market situation, this is the point at which the purchaser incurs a liability to pay.

A service is acquired at the time that the producer provides it, no change of ownership being involved.
The time at which acquisitions are recorded, and their prices, should also be consistent with the way in which the same transac- tions are recorded in the expenditure data used for weighting purposes.

1.152 The time at which payment is made may be determined mainly by institutional arrangements and administrative convenience.
When payments are not made in cash, there may be a signiﬁcant lapse of time before the consumer’s bank account is debited for a 17 CONSUMER PRICE INDEX MANUAL: THEORY AND PRACTICE purchase paid for by cheque, by credit card or similar arrangements.

The time at which these debits are even- tually made is irrelevant for the recording of the acqui- sitions and the prices.
On the other hand, when the acquisition of a good or service is ﬁnanced by the cre- ation of a new ﬁnancial asset at the time of acquisition, such as a loan to the purchaser, two economically separate transactions are involved, the purchase/sale of the good or service and the creation of the asset.

The price to be recorded is the price payable at the time of acquisition, however the purchase is ﬁnanced.

Of course, the provision of ﬁnance may aﬀect the price payable.
The subsequent repayments of any debt incurred by the pur- chaser and the associated interest payments are ﬁnancial transactions that are quite distinct from the purchase of the good or service whose price has to be recorded.

The explicit or implicit interest payments payable on the amount depend on the capital market, the nature of the loan, its duration, the creditworthiness of the purchaser, and so on.

These points are explained in more detail in Chapter 3.
1.153 The distinction between the acquisition and the use of a consumer good or service outlined above has led to two diﬀerent concepts of a CPI being proposed: A CPI may be intended to measure the average change between two time periods in the prices of the consumer goods and services acquired by households.
Alternatively, a CPI may be intended to measure the average change between two time periods in the prices of the consumer goods and services used by house- holds to satisfy their needs and wants.

The distinction between time of acquisition and time of use is particularly important for durable goods and certain kinds of services.

1.154 Durable and non-durable goods.

A ‘‘non- durable’’ good might be better described as a single use good.
For example, food or drink are used once only to satisfy hunger or thirst.

Many so-called non-durable consumer goods are in fact extremely durable physically.

Households may hold substantial stocks of non- durables, such as many foodstuﬀs and fuel, for long periods of time before they are used.

1.155 The distinguishing feature of a durable con- sumption good is that it is durable under use.
Consumer durables can be used repeatedly or continuously to satisfy the needs or wants of consumers over a long period of time, possibly many years: for example, fur- niture or vehicles.

For this reason, a durable is often described as providing a ﬂow of services to the consumer over the period it is used (see also Box 14.3 of Chapter 14).

There is a close parallel between the deﬁnitions of consumer durables and ﬁxed assets.
Fixed assets are deﬁned in national accounts as goods that are used repeatedly or continuously over long periods of time in processes of production: for example, buildings or other structures, machinery and equipment.

1.156 A list of the diﬀerent kinds of consumer dura- bles distinguished in the Classiﬁcation of Individual Consumption according to Purpose (COICOP) is given in Chapter 3.
Of course, some durables last much longer than others, the less durable ones being described as 18 ‘‘semi-durables’’ in COICOP: for example, clothing.

It should be noted that dwellings are classiﬁed as ﬁxed assets, not durable consumption goods, and are there- fore not included in COICOP.

Dwellings are used to produce housing services.

These services are consumed by tenants or owner-occupiers, as the case may be, and are therefore included in COICOP.
1.157 Many services are durable and are also not fully consumed, or used up, at the time they are acquired.

Some services bring about long-lasting improvements from which the consumers derive enduring beneﬁts.

The condition and quality of life of persons receiving med- ical treatments such as hip replacements or cataract surgery, for example, are substantially and permanently improved.

Similarly, consumers of educational services can derive lifetime beneﬁts from them.
Expenditures on education and health also share with durable goods the characteristic that they are also often so costly that they have to be ﬁnanced by borrowing or by running down other assets.

1.158 Expenditures on durable goods and durable services are liable to ﬂuctuate, whereas using up such goods and services is likely to be a fairly steady process.

However, the using up cannot be directly observed and valued.
It can only be estimated by making assumptions about the timing and duration of the ﬂows of beneﬁts.

Partly because of the conceptual and practical diﬃcul- ties involved in measuring uses, statistical oﬃces tend to adopt the acquisitions approach to consumer durables in both their national accounts and CPIs.

1.159 A consumer price index based on the acquisi- tions approach.

Households may acquire goods and services for purposes of consumption in four main ways.
They may: – purchase them in monetary transactions; – produce them themselves for their own consumption; – receive them as payments in kind in barter transac- tions, particularly as remuneration in kind for work done; – receive them as free gifts, or transfers, from other economic units.

1.160 The broadest possible scope for goods and services based on the acquisitions approach would be one covering all four categories, irrespective of who bears the costs.
It would therefore include all social transfers in kind in the form of education, health, housing and other goods and services provided free of charge, or at nomi- nal prices, to individual households by governments or non-proﬁt institutions (NPIs).

Total acquisitions are equivalent to the total actual individual consumption of (non-institutional) households, as deﬁned in the SNA (see Chapter 14).
Collective services provided by govern- ments to the community as whole, such as public admin- istration and defence, are not included and are outside the scope of a CPI.

1.161 From the point of view of the government or NPI that provides and pays for them, social transfers are valued either by the market prices paid for them or by the costs of producing them.

From the point of view of the receiving households they have zero or nominal prices.
For CPI purposes, the appropriate price is that paid AN INTRODUCTION TO CONSUMER PRICE INDEX METHODOLOGY by the household.

The price paid by the government belongs in a price index for government expenditures.

When households incur zero expenditures, the services provided free carry zero weight in a CPI.
However, when governments and NPIs introduce charges for goods or services that were previously provided free, the increase from a zero to a positive price could be captured by a CPI, as explained in Chapter 3.

1.162 Expenditures versus acquisitions.

Expenditures need to be distinguished from acquisitions.

Expenditures are incurred by the economic units that bear the costs.
Households do not incur expenditures on social trans- fers in kind, so the scope of households’ expenditures is generally narrower than the scope of their acquisitions.

Moreover, not all expenditures are monetary.

A mone- tary expenditure occurs when a household pays in cash, by cheque or credit card, or otherwise incurs a ﬁnancial liability to pay.

Only monetary expenditures generate monetary prices that can be observed and recorded for CPI purposes.
1.163 Non-monetary expenditures occur when house- holds pay, but in other ways than cash.

There are three important categories of non-monetary expenditures: In barter transactions, households exchange con- sumption goods and services among themselves.

As the values of the goods and services surrendered as payments constitute negative expenditures, the expen- ditures should cancel out so that barter transactions between households carry zero weight on aggregate.
They can be ignored in practice for CPI purposes.

 When employees are remunerated in kind, they pur- chase the goods or services, but pay with their labour, not cash.

Monetary values can be imputed for the expenditures implicitly incurred by the households.

 Similarly, when households produce goods and ser- vices for themselves, they incur the costs, some of which may be monetary in the form of purchased inputs.
The monetary values of the implicit expendi- tures on the outputs produced can be imputed on the basis of the corresponding market prices.

If such imputed prices were to be included in the CPI, the prices of the inputs would have to be excluded to avoid double counting.

1.164 A hierarchy of consumption aggregates.


A hierarchy of possible consumption aggregates may be envisaged, as explained in Chapter 14: – total acquisitions of goods and services by house- holds; – less social transfers in kind=households’ total expenditures; – less non-monetary expenditures=households’ mone- tary expenditures.

The choice of consumption aggregate is a policy matter.
For example, if the main reason for compiling a CPI is to measure inﬂation, the scope of the index might be restricted to household monetary expenditures on con- sumption, inﬂation being essentially a monetary phe- nomenon.

Prices cannot be collected for the consumer goods and services involved in non-monetary expendi- tures, although they can be estimated on the basis of the prices observed in corresponding monetary transactions.
The European Union’s Harmonized Indices of Con- sumer Prices, which are speciﬁcally intended to measure inﬂation within the EU, are conﬁned to monetary expenditures.

Unconditional and conditional cost of living indices 1.165 Cost of living indices, or COLIs, are explained in Chapters 15 and 17.

As also noted in Chapter 3, the scope of a COLI depends on whether it is conditional or unconditional.
The welfare of a household depends not only on the utility derived from the goods and services it consumes, but on the social, political and physical environment in which the household lives.
An uncondi- tional cost of living index measures the change in the minimum cost of maintaining a given level of welfare in response to changes in any of the factors that aﬀect welfare, whereas a conditional cost of living index mea- sures the change in the minimum cost of maintaining a given level of utility or welfare resulting from changes in consumer prices, holding the environmental factors constant.
1.166 An unconditional COLI may be a more comprehensive cost of living index than a conditional COLI, but it is not a more comprehensive price index.

An unconditional index does not include any more price information than a conditional index and it does not give more insight into the impact of price changes on welfare.

On the contrary, the impact of the price changes is diluted and obscured the more environmental vari- ables are included within the scope of an unconditional index.
In order to qualify as a price index, a COLI must be conditional.

Speciﬁc types of transactions 1.167 Given that conceptually, a CPI is an index that measures changes in the prices of consumption goods and services, expenditures on items that are not consumption goods and services fall outside the scope of the CPI; for example, expenditures on assets such as land or bonds, shares and other ﬁnancial assets.
Simi- larly, payments that do not involve any ﬂows of goods or services in return for the payments are outside the scope; for example, payments of income taxes or social security contributions.

1.168 Transfers.

A transfer occurs when one eco- nomic unit provides a good, service or asset, including money, to another without receiving any counterpart good, service or asset in return.

As no good or service is acquired when a household makes a transfer, the transfer must be outside the scope.
For this reason, compulsory cash transfers, such as payments of direct taxes on income or wealth, must be outside the scope of a CPI.

It is not always clear, however, whether certain payments to government are transfers or purchases of services.

For example, payments to obtain certain kinds of licences are sometimes taxes under another name, whereas in other cases the government may provide a service by exercising some kind of supervisory, reg- ulatory or control function.
Gifts or donations must be 19 CONSUMER PRICE INDEX MANUAL: THEORY AND PRACTICE transfers and therefore outside the scope.

On the other hand, subscriptions to clubs and societies which provide their members with some kind of service in return are included.

Tips and gratuities can be borderline cases.

When they are eﬀectively an expected, even obligatory, part of the payment for a service they are not transfers and should be treated as part of the price paid.
1.169 Undesirable or illegal goods or services.

All goods and services that households willingly buy on the market to satisfy their own needs and wants should be included, even if most people might regard them as undesirable or even if they are prohibited by law.

Of course, illegal goods and services may have to be ex- cluded in practice because the requisite data cannot be collected.

1.170 Financial transactions.


Financial transactions occur when one kind of ﬁnancial asset is exchanged for another, bearing in mind that money is itself a ﬁnancial asset.

For example, the purchase of a bond or share is a ﬁnancial transaction.

Borrowing is a ﬁnancial transac- tion in which cash is exchanged, the counterpart being the creation of a ﬁnancial asset or liability.
1.171 No consumption occurs when a ﬁnancial transaction takes place, even though ﬁnancial transac- tions may be undertaken in order to facilitate future consumption.

Financial transactions as such are not covered by CPIs because, by deﬁnition, no goods are exchanged, nor services provided, in ﬁnancial transac- tions.

However, some ‘‘ﬁnancial’’ transactions may not be entirely ﬁnancial because they may include an explicit or implicit service charge in addition to the provision of an asset, such as a loan.
As a service charge constitutes the purchase of a service by the household, it should be included in a CPI, although it may be diﬃcult to separate out the service charge in some cases.

For example, foreign exchange transactions are ﬁnan- cial transactions in which one ﬁnancial asset is ex- changed for another.

Changes in the price of a foreign currency in terms of the domestic currency resulting from changes in the exchange rate are outside the scope of a CPI.
On the other hand, the commission charges associated with the exchange of currencies are included as payments for the services rendered by the foreign exchange dealers.

1.172 Households may borrow in order to make large expenditures on durables or houses, but also to ﬁnance large educational or health expenses, or even expensive holidays.

Whatever the purpose of the bor- rowing, the ﬁnancial transaction in which the loan is contracted is outside the scope of a CPI.
The treatment of the interest payable on loans is a separate issue con- sidered below.

1.173 Composite transactions.

As just noted, some transactions are composite transactions containing two or more components whose treatment may be quite diﬀerent for CPI purposes.

For example, part of a life insurance premium is a ﬁnancial transaction leading to the creation of a ﬁnancial claim and is therefore outside the scope, whereas the remainder consists of a service charge which should be covered by a CPI.
The two components are not separately itemized, however.
20 1.174 As explained in Chapter 3, the treatment of payments of nominal interest is diﬃcult because it may have four conceptually quite diﬀerent components: – a pure interest payment; – a risk premium that depends on the creditworthiness of the borrower; – a service charge payable to the bank, moneylender or other ﬁnancial institution engaged in the business of making loans; – a payment to compensate the creditor for the real holding loss incurred on the principal of the loan during inﬂation.
The fourth component is clearly outside the scope of a CPI as it is a capital ﬂow.

Conversely, the third com- ponent, the service charge, should clearly be included.

The treatment of the ﬁrst two components is con- troversial.

When there is signiﬁcant inﬂation or a very imperfect capital market, payments of nominal interest may be completely dominated by the last two compo- nents, both of which are conceptually quite diﬀerent from the concept of pure interest.
For example, the ‘‘interest’’ charged by a village moneylender may be mostly a high service charge.

It may be impossible to decompose the various components of nominal interest in practice.

The treatment of nominal interest as a whole remains diﬃcult and somewhat controversial.

Household production 1.175 When households engage in production for the market, the associated business transactions are all outside the scope of a CPI.
Expenditures incurred for business purposes are excluded, even though they involve purchases of goods and services that might be used to satisfy the personal needs and wants of members of the household instead.

1.176 Households also produce goods and services for their own consumption, mainly service production such as the preparation of meals, the care of children, the sick or the elderly, the cleaning and maintenance of durables and dwellings, the transportation of household members, and so on.
Owner-occupiers produce housing services for their own consumption.

Households also grow vegetables, fruit, ﬂowers or other crops for their own use.

1.177 Many of the goods or services purchased by households do not provide utility directly but are used as inputs into the production of other goods and services that do provide utility: for example, raw foodstuﬀs, fertilizers, cleaning materials, paints, electricity, coal, oil, petrol, and so on.
1.178 In principle, a CPI should record changes in the prices of the outputs from these production activities, as it is the outputs rather than the inputs that are actually consumed and provide utility.

However, as the outputs are not themselves purchased, no prices can be observed for them.

Prices could be imputed for them equal to the prices they would fetch on the market, but this would make a CPI heavily dependent on assumed rather than collected prices.
The pragmatic solution recommended in Chapter 3 is to treat all goods and services purchased on AN INTRODUCTION TO CONSUMER PRICE INDEX METHODOLOGY the market to be used exclusively as inputs into the production of other goods and services that are directly consumed by households as if they were themselves consumption goods and services.

On this principle, goods such as insecticides and electricity are treated as providing utility indirectly and included in CPIs.
This is, of course, the solution usually adopted in practice not only for CPIs but also in national accounts, where most expenditures on inputs into household production are classiﬁed as ﬁnal consumption expenditures.

1.179 In some countries, there is an increasing ten- dency for households to purchase prepared, take-away meals rather than the ingredients.
As the prices of such meals cost more than the sum of the ingredients that the households previously purchased, the weight attached to food consumption increases.

This partly reﬂects the fact that the costs of the households’ own labour inputs into the preparation of meals were previously ignored.

Var- ious kinds of household service activities that were pre- viously outside the scope of a CPI may be brought within the scope if households choose to pay others to perform the services.
1.180 Subsistence agriculture and owner-occupied housing.

In the case of two important types of production for own consumption within households, namely agri- cultural production for own consumption and housing services produced by owner-occupiers, the national ac- counts do actually try to record the values of the outputs produced and consumed rather than the inputs.

Simi- larly, CPIs may also try to price the outputs rather than the inputs in these two cases.
1.181 In principle, the prices of the outputs from own-account agricultural production may be included in CPIs, even though they are imputed.

On the other hand, for households relying on subsistence agriculture, the prices of inputs of agricultural materials purchased on the market may be their main exposure to inﬂation.

Two points may be noted.


First, the imputed market value of the output should usually be greater than the costs of the purchased inputs, if only because it should cover the costs of the labour inputs provided by the household.

Thus, pricing the purchased inputs rather than the outputs may mean that the consumption of own agricultural production in CPIs does not receive suf- ﬁcient weight.

Second, double counting should be avoi- ded.


If the imputed prices of the outputs are included, the actual prices of the inputs consumed should not be included as well.

1.182 In the case of owner-occupied housing, the situation is complicated by the fact that the production requires the use of the capital services provided by a large ﬁxed asset in the form of the dwelling itself.
Even if the inputs into the production of housing services are priced for CPI purposes, it is still necessary to impute prices for the inputs of capital services (mainly depre- ciation plus interest) provided by the dwelling.

Some countries therefore prefer to impute the prices of the outputs of housing services actually consumed on the basis of the rents payable for the same kind of dwellings rented on the market.
The treatment of owner-occupied housing is complex, and somewhat controversial, and is considered in Chapters 3, 9, 10 and 23, among others.

Coverage of households and outlets 1.183 As explained in Chapter 3, households may be either individual persons or groups of persons living together who make common provision for food or other essentials for living.
A CPI may be required to cover: – either the consumption expenditures made by house- holds resident in a particular area, usually a country or region, whether the expenditures are made inside or outside the area – this is called the ‘‘national’’ concept of expenditure; – or the consumption expenditures that take place within a particular area, whether made by households resident in that area or residents of other areas – this is called the ‘‘domestic’’ concept.
Adopting the domestic concept may make it more dif- ﬁcult to collect the relevant disaggregated expenditure data in household surveys.

A CPI may also be deﬁned to cover a group of countries, such as the European Union.

1.184 Not all kinds of households have to be in- cluded.

As explained in Chapter 3, some countries choose to exclude particular categories of households such as very wealthy households or households engaged in agricul- ture.
Some countries also compile diﬀerent indices de- signed to cover diﬀerent groups of households, such as households resident in diﬀerent regions.

Another possi- bility is to compile a general CPI designed to cover all or most households and, in addition, one or more special indices aimed at particular sections of the community, such as households headed by pensioners.

The precise coverage of households is a matter of choice.
It is inev- itably inﬂuenced by what are believed to be the main uses of the index.

The set of households actually covered by the CPI is described as the ‘‘reference population’’.

Price variation 1.185 Prices for exactly the same good or service may vary between diﬀerent outlets, while diﬀerent prices may sometimes be charged to diﬀerent types of custo- mers.

Prices may also vary during the course of the month to which the index relates.
Conceptually, it is necessary to distinguish such pure price variation from price diﬀerences that are attributable to diﬀerences in the quality of the goods or services oﬀered, although it is not always easy to distinguish between the two in practice.

The existence of pure price diﬀerences reﬂects some form of market imperfections, such as consumers’ lack of information or price discrimination.
1.186 When pure price diﬀerences exist, a change in market conditions may make it possible for some households to switch from purchasing at higher prices to purchasing at lower prices, for example if new outlets open that oﬀer lower prices.

The resulting fall in the average price paid by households counts as a price fall for CPI purposes, even though the price charged by each individual outlet may not change.
If the prices are col- lected from the outlets and switches in households’ purchasing habits remain unobserved, the CPIs are said to be subject to outlet substitution bias, as explained in more detail in Chapter 11.
On the other hand, when the price diﬀerences reﬂect diﬀerences in the quality of the 21 CONSUMER PRICE INDEX MANUAL: THEORY AND PRACTICE goods and services sold in the diﬀerent outlets, switching from outlets selling at higher prices to outlets selling at lower prices simply means that households are choosing to purchase lower-quality goods or services.

In itself, this does not imply any change in price.
Classiﬁcations 1.187 As explained in Chapter 3, the classiﬁcation of household expenditures used in a CPI provides the nec- essary framework for the various stages of CPI compi- lation.

It provides a structure for purposes of weighting and aggregation, and also a basis for stratifying the samples of products whose prices are collected.
The goods and services covered by a CPI may be classiﬁed in several ways: not simply on the basis of their physical characteristics but also by the purposes they serve and the degree of similarity of their price behaviour.

Product-based and purpose-based classiﬁcations diﬀer but can usually be successfully mapped onto each other.

In practice, most countries use a hybrid classiﬁcation system in which the breakdown at the highest level is by purpose while the lower-level breakdowns are by product type.
This is the case for the recently revised internationally agreed Classiﬁcation of Individual Con- sumption according to Purpose (COICOP), which pro- vides a suitable classiﬁcation for CPI purposes.

1.188 The ﬁrst level of classiﬁcation in COICOP consists of 12 divisions covering total consumption ex- penditures of households.

As just noted, the breakdown into divisions is essentially by purpose.
At the second level of disaggregation, the 12 divisions are divided into 47 groups of products, which are in turn divided into 117 classes of products at the third level.

Chapter 3 provides a listing of ten classes of goods deﬁned as durables in COICOP.

It also gives a list of seven classes described as semi-durables, such as clothing, footwear and house- hold textiles.

1.189 The 117 classes at the lowest level of aggrega- tion of COICOP are not suﬃciently detailed for CPI purposes.
They can be divided into sub-classes using the sub-classes of the internationally agreed Central Product Classiﬁcation (CPC).

Even some of these may require further breakdown in order to arrive at some of the ele- mentary aggregates used for CPI purposes.

In order to be useful for CPI purposes, expenditure weights must be available for the various sub-classes or elementary aggre- gates.
From a sampling perspective, it is desirable for the price movements of the individual products within the elementary aggregates to be as homogeneous as pos- sible.

The elementary aggregates may also be divided into strata for sampling purposes, on the basis of location or the type of outlet in which the products are sold.
Consumer price indices and national accounts price deﬂators 1.190 Appendix 3.1 of Chapter 3 explains the dif- ferences between the overall CPI and the deﬂator for total household consumption expenditures in national accounts.

In practice, CPIs may be designed to cover only a subset of the households and a subset of the 22 expenditures covered by the national accounts.

More- over, the index number formulae needed for CPIs and national accounts deﬂators may be diﬀerent.
These dif- ferences mean that the overall CPI is generally not the same as the deﬂator for total household consumption expenditures in the national accounts.

On the other hand, the basic price and expenditure data collected and used for CPI purposes are also widely used to build up the price indices needed to deﬂate the individual com- ponents of household consumption in the national accounts.

Expenditure weights 1.191 As already noted, there are two main stages in the calculation of a CPI.
The ﬁrst is the collection of the price data and the calculation of the elementary price indices.

The second is the averaging of the elementary price indices to arrive at price indices at higher levels of aggregation up to the overall CPI itself.

Expenditure data are needed for the elementary aggregates that can be used as weights in the second stage.

These weights are needed whatever index number formula is used for aggregation purposes.
Chapter 4 is concerned with the derivation, and sources, of the expenditure weights.

Household expenditure surveys and national accounts 1.192 The principal data source for household con- sumption expenditures in most countries is a household expenditure survey (HES).

An HES is a sample survey of thousands of households that are asked to keep rec- ords of their expenditures on diﬀerent kinds of con- sumer goods and services over a speciﬁed period of time, such as a week or longer.
The size of the sample obviously depends on the resources available, but also on the extent to which it is desired to break down the survey results by region or type of household.

HESs are costly operations.

This manual is not concerned with the conduct of HESs or with general sampling survey tech- niques or procedures.

There are several standard texts on survey methods to which reference may be made.
Household expenditure surveys may be taken at speci- ﬁed intervals of time, such as every ﬁve years, or they may be taken each year on a continuing basis.

1.193 HESs can impose heavy burdens on the respondents, who have to keep detailed expenditure records of a kind that they would not normally keep, although this may become easier when supermarkets or other retail outlets provide detailed printouts of pur- chases.

HESs tend to have some systematic biases.


For example, many households either deliberately, or uncon- sciously, understate the amounts of their expenditures on certain ‘‘undesirable’’ products, such as gambling, alco- holic drink, tobacco or drugs.

Corrections can be made for such biases.

Moreover, the data collected in HESs may also need to be adjusted to bring them into line with the concept of expenditure required by the CPI.
For example, the imputed expenditures on the housing ser- vices produced and consumed by owner-occupiers are not collected in HESs.

AN INTRODUCTION TO CONSUMER PRICE INDEX METHODOLOGY 1.194 As explained in Chapter 14, the use of the commodity ﬂow method within the supply and use tables of the SNA enables data drawn from diﬀerent primary sources to be reconciled and balanced against each other.
The commodity ﬂow method may be used to improve estimates of household consumption expendi- tures derived from expenditure surveys by adjusting them to take account of the additional information provided by statistics on the sales, production, imports and exports of consumer goods and services.
By draw- ing on various sources, the household expenditure data in the national accounts may provide the best estimates of aggregate household expenditures, although the classiﬁcations used may not be ﬁne enough for CPI purposes.
Moreover, because HESs may be conducted only at intervals of several years, the expenditure data in the national accounts may be more up to date, as national accounts are able to draw upon other kinds of more recent data, such as retail sales and the production and import of consumer goods and services.

It is impor- tant to note, however, that national accounts should not be viewed as if they were an alternative, indepen- dent data source to HESs.
On the contrary, HESs pro- vide one of the main sources for the expenditure data on household consumption used to compile national accounts.

surveys 1.195 Household expenditure in many countries may not be conducted as frequently as might be desired for CPI, or national accounts, purposes.

National HESs can be very costly and onerous for the households, as already noted.

They may be conducted only once every ﬁve or ten years, or even at longer intervals.
In any case, conducting and processing HESs is time-consuming, so the results may not be available for CPI purposes until one or two years after the surveys have been conducted.

It is for these practical reasons that CPIs in many countries are Lowe indices that use the quantities of some base period b that may precede the time reference period 0 by a few years and period t by many years.
1.196 Some countries conduct continuous HESs not only in order to update their CPI weights but also to improve their national accounts.

Of course, the same panel of households does not have to be retained indef- initely; the panel can be gradually rotated by dropping some households and replacing them by others.

Coun- tries that conduct continuous expenditure surveys are able to revise and update their expenditure weights each year so that the CPI becomes a chain index with annual linking.
Even with continuous expenditure surveys, however, there is a lag between the time at which the data are collected and the time at which the results are pro- cessed and ready for use, so that it is never possible to have survey results that are contemporaneous with the price changes.

Thus, even when the weights are updated annually, they still refer to some period that precedes the time reference period.
For example, when the price reference period is January 2000, the expenditure weights may refer to 1997 or 1998, or both years.

When the price reference period moves forward to January 2001, the weights move forward to 1998 or 1999, and so on.

Such an index is a chain Lowe index.


1.197 Some countries prefer to use expenditure weights that are the average rates of expenditure over periods of two or three years in order to reduce ‘‘noise’’ caused by errors of estimation (the expenditure surveys are only samples) or erratic consumer behaviour over short periods of time resulting from events such as booms or recessions, stock market ﬂuctuations, oil shocks, or natural or other disasters.
Other sources for estimating expenditure weights 1.198 If expenditures need to be disaggregated by region for sampling or analytical purposes, it is possible to supplement whatever information may be available by region in HESs by using data from population cen- suses.

Another data source may be food surveys.

These are special surveys, conducted in some countries, that focus on households’ expenditures on food products.
They can provide more detailed information on food expenditures than that available from HESs.

1.199 Another possible source of information con- sists of points of purchase (POP) surveys, which are conducted in some countries.

A POP survey is designed to provide information about the retail outlets at which households purchase speciﬁed groups of goods and ser- vices.

Households are asked, for each item, about the amounts spent in each outlet and the names and addresses of the outlets.
The main use for a POP survey is for selecting the sample of outlets to be used for price collection purposes.

Collection of price data 1.200 As explained in Chapter 9, there are two levels of calculation involved in a CPI.

At the lower level, samples of prices are collected and processed to obtain lower-level price indices.

These lower-level indices are the elementary indices, whose properties and behaviour are studied in Chapter 20.
At the higher level, the ele- mentary indices are averaged to obtain higher-level indices using expenditures as weights.

At the higher level, all the index number theory elaborated in Chap- ters 15 to 18 comes into play.

1.201 Lower-level indices are calculated for ele- mentary aggregates.

Depending on the resources avail- able and procedures adopted by individual countries, these elementary aggregates could be sub-classes or micro-classes of the expenditure classiﬁcation described above.
If it is desired to calculate CPIs for diﬀerent regions, the sub-classes or micro-classes have to be divided into strata referring to the diﬀerent regions.

In addition, in order to improve the eﬃciency of the sam- pling procedures used to collect prices, it will usually be desirable, if feasible, to introduce other criteria into the deﬁnitions of the strata, such as the type of outlet.
When the sub-classes or micro-classes are divided into strata for data collection purposes, the strata themselves become the elementary aggregates.

As a weight needs to be attached to each elementary aggregate in order to indices, an estimate of the calculate the higher-level expenditure within each elementary aggregate must be 23 CONSUMER PRICE INDEX MANUAL: THEORY AND PRACTICE available.
Expenditure or quantity data are typically not available within an elementary aggregate, so the ele- mentary indices have to be estimated from price data alone.

This may change if scanner data from electronic points of sale become more available.

1.202 Chapter 5 is concerned with sampling strate- gies for price collection.

Chapter 6 is concerned with the methods and operational procedures actually used to collect prices.
In principle, the relevant prices for a CPI should be the purchasers’ prices actually paid by households, but it is generally neither practical nor cost- eﬀective to try to collect prices each month or quarter directly from households, even though expenditure data are collected directly from households in household expenditure surveys.
In practice, the prices that are collected are not actual transaction prices, but rather the prices at which goods and services are oﬀered for sale in outlets such as retail shops, supermarkets or service providers.

However, it may become increasingly feasible to collect actual transactions prices as more goods and services are sold through electronic points of sale that record both prices and expenditures.
Random sampling and purposive sampling 1.203 Given that the prices are collected from the sellers, there are two diﬀerent sampling problems that arise.

The ﬁrst is how to select the individual products within an elementary aggregate whose prices are to be collected.

The second is how to select a sample of outlets selling those products.

For some products, it may not be necessary to visit retail outlets to collect prices because there may be only a single price applying throughout the country.
Such prices may be collected from the central organization responsible for ﬁxing the prices.

The fol- lowing paragraphs refer to the more common situation in which prices are collected from a large number of outlets.

1.204 As explained in Chapter 5, the universe of products from which the sample is taken has several dimensions.
The products may be classiﬁed not only on the basis of the characteristics and functions that determine their place in COICOP, but also according to the locations and outlets at which they are sold and the times at which they are sold.

The fact that the universe is continually changing over time is a major problem, not only for CPIs but also for most other economic statis- tics.

Products disappear to be replaced by other kinds of products, while outlets close and new ones open.
The fact that the universe is changing over time creates both conceptual and practical problems, given that the mea- surement of price changes over time requires some continuity in the products priced.

In principle, the price changes recorded should refer to matched products that are identical in both time periods.

The problems created when products are not identical are considered in some detail later.
1.205 In designing the sample for price collection purposes, due attention should be paid to standard sta- tistical criteria to ensure that the resulting sample esti- mates are not only unbiased and eﬃcient in a statistical sense, but also cost-eﬀective.

There are two types of bias 24 encountered in the literature on index numbers, namely sampling bias as understood here and the non-sampling biases in the form of substitution bias or representativity bias, as discussed in Chapter 10.
It is usually clear from the context which type of bias is meant.

1.206 There is a large literature on sampling survey techniques to which reference may be made and which need not be summarized here.

In principle, it would be desirable to select both outlets and products using ran- dom sampling with known probabilities of selection.

This ensures that the sample of products selected is not distorted by subjective factors and enables sampling errors to be calculated.
Many countries nevertheless continue to rely heavily on the purposive selection of outlets and products, because random sampling may be too diﬃcult and too costly.

Purposive selection is be- lieved to be more cost-eﬀective, especially when the sampling frames available are not comprehensive and not well suited to CPI purposes.
It may also be cost- eﬀective to collect a ‘‘cluster’’ of prices on diﬀerent products from the same outlet, instead of distributing the price collection more thinly over a larger number of outlets.

1.207 Eﬃcient sampling, whether random or pur- posive, requires comprehensive and up-to-date sampling frames.

Two types of frames are needed for CPI pur- poses: one listing the universe of outlets, and the other listing the universe of products.
Examples of possible sampling frames for outlets are business registers, cen- tral or local government administrative records or tele- phone directories.

When the sampling frames contain the requisite information, it may be possible to increase eﬃciency by selecting samples of outlets using prob- abilities that are proportional to the size of some rele- vant economic characteristic, such as the total value of sales.

Sampling frames for products are not always readily available in practice.
Possible frames are cata- logues or other product lists drawn up by major man- ufacturers, wholesalers or trade associations, or lists of products that are speciﬁc to individual outlets such as large supermarkets.

1.208 Depending on the information available in the sampling frame, it may be possible to group the outlets into strata on the basis of their location and size, as indicated by sales or employees.
When there is infor- mation about size, it may be possible to increase eﬃ- ciency by taking a random sample of outlets with probabilities proportional to size.

In practice, however, there is also widespread use of purposive sampling.

1.209 In most countries, the selection of most of the individual items to be priced within the selected outlets tends to be purposive, being speciﬁed by the central oﬃce responsible for the CPI.
The central oﬃce draws up lists of products that are deemed to be representative of the products within an elementary aggregate.

The lists can be drawn up in collaboration with managers of wholesale or large retail establishments, or other experts with practical experience and knowledge.

The actual procedures are described in more detail in Chapter 6.
1.210 It has been argued that the purposive selection of products is liable to introduce only a negligible amount of sampling bias, although there is not much AN INTRODUCTION TO CONSUMER PRICE INDEX METHODOLOGY conclusive evidence on this matter.

In principle, random sampling is preferable and it is also quite feasible.

For example, the United States Bureau of Labor Statistics makes extensive use of random selection procedures to select both outlets and products within outlets.
When the selection of products is delegated to the individual price collectors, it is essential to ensure that they are well trained and briefed, and closely supervised and monitored.

Methods of price collection 1.211 The previous section focused on the sampling issues that arise when prices have to be collected for a large number of products from a large number of out- lets.

This section is concerned with some of the more operational aspects of price collection.

1.212 Central price collection.


Many important prices can be collected directly by the central oﬃce responsible for the CPI from the head oﬃce of the or- ganization responsible for ﬁxing the prices.

When prices are the same throughout the country, collection from individual outlets is superﬂuous: Some tariﬀs or service charges are ﬁxed nationally and apply throughout the country.
This may be the case for public utilities such as water, gas and electricity, postal services and telephone charges, or public transport.

The prices or charges can be obtained from the rele- vant head oﬃces.

 Some national chains of stores or supermarkets may charge the same prices everywhere, in which case the prices can be obtained from their head oﬃces.
Even when national chains do not charge uniform prices, there may be only a few minor regional diﬀerences in the prices and all the relevant information may be obtainable centrally.

 Many of these prices determined centrally may change very infrequently, perhaps only once or twice or year, so they do not have to be collected monthly.

More- over, many of these prices can be collected by tele- phone, fax or email and may not require visits to the head oﬃces concerned.

1.213 Scanner data.
1.213 Scanner data.

One important new develop- ment is the increasing availability in many countries of large amounts of very detailed ‘‘scanner’’ data obtained from electronic points of sale.

Such data are collated by commercial databases.

Scanner data are up to date and comprehensive.

An increasingly large proportion of all goods sold are being scanned as they pass through electronic points of scale.
1.214 The potential beneﬁts of using scanner data are obviously considerable and could ultimately have a signiﬁcant impact on the way in which price data are collected for CPI purposes.

Not enough experience is yet available to provide general guidelines about the use of scanner data.

Clearly, statistical oﬃces should monitor developments in this ﬁeld closely and explore the pos- sibility of exploiting this major new source of data.
Scanner data also increase the scope for using improved methods of quality adjustment, including hedonic meth- ods, as explained in Chapter 7.

1.215 Local price collection.

When prices are col- lected from local outlets, the individual products selected for pricing can be determined in two ways.

One way is for a speciﬁc list of individual products to be determined in advance by the central oﬃce responsible for the CPI.
Alternatively, the price collector can be given the dis- cretion to choose from a speciﬁed range of products.

The collector may use some kind of random selection procedure, or select the products that sell the most or are recommended by the shop owner or manager.

An individual product selected for pricing in an individual outlet may be described as a sampled product.

It may be a good or a service.


1.216 When the list of products is determined in advance by the central oﬃce, the objective is usually to select products that are considered to be representative of the larger group of products within an elementary aggregate.

The central oﬃce also has to decide how loosely or tightly to describe, or specify, the repre- sentative products selected for pricing.
In theory, the number of diﬀerent products that might be identiﬁed is to some extent arbitrary, depending on the number of economic characteristics that are deemed to be relevant or important.

For example, ‘‘beef ’’ is a generic term for a group of similar but nevertheless distinct products.

There are many diﬀerent cuts of beef, such as minced beef, stewing steak or rump steak, each of which can be considered a diﬀerent product and which can sell at very diﬀerent prices.
Furthermore, beef can also be classiﬁed according to whether it is fresh, chilled or frozen, and cross-classiﬁed again according to whether it comes from domestic or imported animals, or from animals of diﬀerent ages or breeds.

1.217 Tightening the speciﬁcations ensures that the central oﬃce has more control over the items actually priced in the outlets, but it also increases the chance that some products may not actually be available in some outlets.
Loosening the speciﬁcations means that more items may be priced but leaves the individual price col- lectors with more discretion with regard to the items actually priced.

This could make the sample as a whole less representative.

Continuity of price collection 1.218 A CPI is intended to measure pure price changes.
The products whose prices are collected and compared in successive time periods should ideally be perfectly matched; that is, they should be identical in respect of their physical and economic characteristics.

When the products are perfectly matched, the observed price changes are pure price changes.
When selecting it is therefore necessary to representative products, ensure that enough of them can be expected to remain on the market over a reasonably long period of time in exactly the same form or condition as when ﬁrst select- ed.

Without continuity, there would not be enough price changes to measure.

1.219 Having identiﬁed the items whose prices are to be collected, the normal strategy is to continue pricing exactly those same items for as long as possible.
Price collectors can do this if they are provided with very precise, or tight, speciﬁcations of the items to be priced.

25 CONSUMER PRICE INDEX MANUAL: THEORY AND PRACTICE Alternatively, they must keep detailed records them- selves of the items that they have selected to price.
1.220 The ideal situation for a price index would be one in which all the products whose prices are being recorded remain on the market indeﬁnitely without any change in their physical and economic characteristics, except of course for the timing of their sale.

It is worth noting that many theorems in index number theory are derived on the assumption that exactly the same set of goods and services is available in both the time periods being compared.
Most products, however, have only a limited economic life.

Eventually, they disappear from the market to be replaced by other products.

As the universe of products is continually evolving, the repre- selected initially may gradually sentative products account for a progressively smaller share of total pur- chases and sales.

As a whole, they may become less and less representative.

As a CPI is intended to cover all products, some way has to be found to accommodate the changing universe of products.
In the case of consumer durables whose features and designs are continually being modiﬁed, some models may have very short lives indeed, being on the market for only a year or less before being replaced by newer models.

1.221 At some point the continuity of the series of price observations may have to be broken.

It may become necessary to compare the prices of some pro- ducts with the prices of other new ones that are very similar but not identical.
Statistical oﬃces must then try to eliminate from the observed price changes the esti- mated eﬀects of the changes in the characteristics of the products whose prices are compared.

In other words, they must try to adjust the prices collected for any changes in the quality of the products priced, as ex- plained in more detail below.
At the limit, a completely new product may appear that is so diﬀerent from those existing previously that quality adjustment is not fea- sible and its price cannot be directly compared with that of any previous product.

Similarly, a product may become so unrepresentative or obsolete that it has to be dropped from the index because it is no longer worth trying to compare its price with those of any of the products that have displaced it.
Resampling 1.222 One strategy to deal with the changing uni- verse of products would be to resample, or reselect, at regular intervals the complete set of items to be priced.

For example, with a monthly index, a new set of items could be selected each January.

Each set of items would be priced until the following January.

Two sets have to be priced each January in order to establish a link between each set of 12 monthly changes.
Resampling each year would be consistent with a strategy of updating the expenditure weights each year.

1.223 Although resampling may be preferable to maintaining an unchanged sample or selection, it is not used much in practice.

Systematically resampling the entire set of products each year would be diﬃcult to manage and costly to implement.
Moreover, it does not provide a complete solution to the problem of the 26 changing universe of products, as it does not capture price changes that occur at the moment of time when new products or new qualities are ﬁrst introduced.

Many producers deliberately use the time when products are ﬁrst marketed to make signiﬁcant price changes.

1.224 A more practical way in which to keep the sample up to date is to rotate it gradually by dropping certain items and introducing new ones.
Items may be dropped for two reasons: The product is believed by the price collector or cen- tral oﬃce to be no longer representative.

It appears to account for a steadily diminishing share of the total expenditures within the basic categories in question.

 The product may simply disappear from the market altogether.

For example, it may have become obsolete as a result of changing technology or unfashionable because of changing tastes, although it could dis- appear for other reasons.
1.225 At the same time, new products or new qual- ities of existing products appear on the market.

At some point, it becomes necessary to include them in the list of items priced.

This raises the general question of the treatment of quality change and the treatment of new products.

Adjusting prices for quality changes 1.226 The treatment of quality change is perhaps the greatest challenge facing CPI compilers.

It is a recurring theme throughout this manual.


It presents both con- ceptual and practical problems for compilers of CPIs.

The whole of Chapter 7 is devoted to the treatment of quality change, while Chapter 8 addresses the closely related topic of new goods and item substitution.
1.227 When a sampled product is dropped from the list of products priced in some outlet, the normal prac- tice is to ﬁnd a new product to replace it in order to ensure that the sample, or selection, of sampled products remains suﬃciently comprehensive and representative.

If the new product is introduced speciﬁcally to replace the old one, it is necessary to establish a link between the series of past price observations on the old item and the subsequent series for the new item.
The two series of observations may, or may not, overlap in one or more periods.

In many cases, there can be no overlap because the new quality, or model, is only introduced after the one which it is meant to replace is discontinued.

Whether or not there is an overlap, the linking of the two price series requires some estimate of the change in quality between the old product and the product selec- ted to replace it.
1.228 However diﬃcult it is to estimate the con- tribution of the change in quality to the change in the observed price, it must be clearly understood that some estimate has to be made either explicitly or, by default, implicitly.

The issue cannot be avoided or bypassed.

All statistical oﬃces have limited resources and many may not have the capacity to undertake the more elaborate explicit adjustments for quality change described in AN INTRODUCTION TO CONSUMER PRICE INDEX METHODOLOGY Chapter 7.
Even though it may not be feasible to under- take an explicit adjustment through lack of data or resources, it is not possible to avoid making some kind of implicit adjustment.

Even apparently ‘‘doing nothing’’ necessarily implies some kind of implicit adjustment, as explained below.

Whatever the resources available to them, statistical oﬃces must be conscious of the impli- cations of the procedures they adopt.
1.229 Three points are stressed in the introductory section of Chapter 7: The pace of innovation is high, and possibly increas- ing, leading to continual changes in the characteristics of products.

 There is not much consistency between countries in the methods they use to deal with quality change.

 A number of empirical studies have demonstrated that the choice of method does matter, as diﬀerent methods can lead to very diﬀerent results.
Evaluation of the effect of quality change on price 1.230 It is useful to try to clarify why one would wish to adjust the observed price change between two items that are similar, but not identical, for diﬀerences in their quality.

A change in the quality of a good or service occurs when there is a change in some, but not most, of its characteristics.

For purposes of a CPI, a quality change must be evaluated from the consumer’s perspec- tive.
As explained in Chapter 7, the evaluation of the quality change is essentially an estimate of the additional amount that a consumer is willing to pay for the new characteristics possessed by the new quality.

This addi- tional amount is not a price increase because it represents the monetary value of the additional satisfaction or utility that is derived from the new quality.
Of course, if the old quality is preferred to the new one, consumers would only be willing to buy the new quality if its price were lower.

1.231 Consider the following hypothetical experi- ment in which a new quality appears alongside an old one.

Assume that the two products are substitutes and that the consumer is familiar with the characteristics of the old and the new qualities.

Use lower case p to refer to prices of the old quality and upper case P for the prices of the new quality.
Suppose that both qualities are oﬀered to the consumer at the same price, namely the price Pt at which the new quality is actually being sold in period t. The consumer is then asked to choose between them and prefers the new quality.

1.232 Suppose next that the price of the old quality is progressively reduced until it reaches p t , at which point the consumer becomes indiﬀerent between pur- chasing the old quality at p t and the new quality at Pt.
Any further decrease below p t causes the consumer to switch back to the old quality.

The diﬀerence between Pt and p is a measure of the additional value that the t consumer places on the new quality as compared with the old quality.

It measures the maximum amount that the consumer is willing to pay for the new quality over and above the price of the old quality.

1.233 Let pt1 denote the actual price at which the old quality was sold in period t1.
For CPI purposes, the price increase between the two qualities is not the observed diﬀerence Pt pt1 but p* t pt1.

It is impor- tant to note that p* t , the hypothetical price for the old quality in period t, is directly comparable with the actual price of the old quality in period t1 because both refer to the same identical product.

The diﬀerence between them is a pure price change.


The diﬀerence between Pt and p* is not a price change but an evaluation of the t diﬀerence in the quality of the two items in period t. The actual price of the new quality in period t needs to be multiplied by the ratio p* t /Pt in order to make the comparison between the prices in periods t1 and t a comparison between products of equal quality in the eyes of the consumer.

The ratio p* t =Pt is the required quality adjustment.
1.234 Of course, it is diﬃcult to estimate the quality adjustment in practice, but the ﬁrst step has to be to clarify conceptually the nature of the adjustment that is required in principle.

In practice, producers often treat the introduction of a new quality, or new model, as a convenient opportunity at which to make a signiﬁcant price change.
They may deliberately make it diﬃcult for consumers to disentangle how much of the observed diﬀerence in price between the old and the new qualities represents a price change.

1.235 Chapter 7 explains the two possibilities open to statistical oﬃces.

One possibility is to make an explicit adjustment to the observed price change on the basis of the diﬀerent characteristics of the old and new qualities.
The other alternative is to make an implicit adjustment by making an assumption about the pure price change; for example, on the basis of price movements observed for other products.

It is convenient to take the implicit methods ﬁrst.

Implicit methods for adjusting for quality changes 1.236 Overlapping qualities.
Suppose that the two qualities overlap, both being available on the market at time t. If consumers are well informed, have a free choice and are collectively willing to buy some of both at the same time, economic theory suggests that the ratio of the prices of the new to the old quality should reﬂect their relative utilities to consumers.

This implies that the diﬀerence in price between the old and the new qualities does not indicate any change in price.
The price changes up to period t can be measured by the prices for the old quality, while the price changes from period t onwards can be measured by the prices for the new quality.

The two series of price changes are linked in period t, the diﬀerence in price between the two qualities not having any impact on the linked series.

1.237 When there is an overlap, simple linking of this kind may provide an acceptable solution to the problem of dealing with quality change.
In practice, however, this method is not used very extensively because the requisite data are seldom available.

Moreover, the conditions may not be consistent with those assumed in the theory.

Even when there is an overlap, consumers may not have had 27 CONSUMER PRICE INDEX MANUAL: THEORY AND PRACTICE time to acquire suﬃcient knowledge of the character- istics to be able to evaluate the relative qualities properly, especially when there is a substantial change in quality.
Not all consumers may have access to both qualities.

When the new quality ﬁrst appears, the market is liable to remain in disequilibrium for some time, as it takes time for consumers to adjust their consumption patterns.

1.238 There may be a succession of periods in which the two qualities overlap before the old quality ﬁnally disappears from the market.
If the market is temporarily out of equilibrium, the relative prices of the two qualities may change signiﬁcantly over time so that the market oﬀers alternative evaluations of the relative qualities depending on which period is chosen.

When new qual- ities that embody major new improvements appear on the market for the ﬁrst time, there is often a tendency for their prices to fall relatively to older qualities before the latter eventually disappear.
In this situation, if the price series for the old and new qualities are linked in a single period, the choice of period can have a substantial eﬀect on the overall change in the linked series.

1.239 The statistician has then to make a deliberate judgement about the period in which the relative prices appear to give the best representation of the relative qualities.
In this situation, it may be preferable to use a more complex linking procedure which uses the prices for both the new and the old qualities in several periods in which they overlap.

However, the information needed for this more complex procedure will never be available if price collectors are instructed only to introduce a new quality when an old one is dropped.

In this case, the timing of the switch from the old to the new can have a signiﬁcant eﬀect on the long-term change in the linked series.
This factor must be explicitly recognized and taken into consideration.

1.240 If there is no overlap between the new and the old qualities, the problems just discussed do not arise as no choice has to be made about when to make the link.

Other and more diﬃcult problems nevertheless take their place.

1.241 Non-overlapping qualities.


In the following sections, it is assumed that the overlap method cannot be used because there is a discontinuity between the series of price observations for the old and new qualities.

Again, using lower case p for the old quality and upper case P for the new, it is assumed that the price data available to the index compiler take the follow- ing form: .

.

.

; pt3; pt2; pt1; Pt; Pt+1; Pt+2; .

.

.
.

.

The problem is to estimate the pure price change between t 1 and t in order to have a continuous series of price observations for inclusion in the index.
Using the same notation as above: – price changes up to period t 1 are measured by the series for the old quality; – the change between t 1 and t is measured by the t is equal to Pt after adjustment t =pt1 where p ratio p for the change in quality; – price changes from period t onwards are measured by the series for the new quality.

28 1.242 The problem is to estimate p t .

This may be done explicitly by one of the methods described later.
Otherwise, one of the implicit methods has to be used.

These may be grouped into three categories: The ﬁrst solution is to assume that p t =pt1=Pt=pt1 or p t =Pt.

No change in quality is assumed to have occurred, so the whole of the observed price increase is treated as a pure price increase.

In eﬀect, this con- tradicts the assumption that there has been a change in quality.

 The second is to assume that p t =pt1.
No price change is assumed to have occurred, the whole of the observed diﬀerence between pt1 and Pt being attributed to the diﬀerence in their quality.

t =pt1=1, or p The third is to assume that p t =pt1=I, where I is an index of the price change for a group of similar pro- ducts, or possibly a more general price index.

1.243 The ﬁrst two possibilities cannot be recom- mended as default options to be used automatically in the absence of any adequate information.
The use of the ﬁrst option can only be justiﬁed if the evidence suggests that the extent of the quality change is negligible, even though it cannot be quantiﬁed more precisely.

‘‘Doing nothing’’, in other words ignoring the quality change completely, is equivalent to adopting the ﬁrst solution.

Conversely, the second can only be justiﬁed if the evi- dence suggests that the extent of any price change between the two periods is negligible.
The third option is likely to be much more acceptable than the other two.

It is the kind of solution that is often used in economic statistics when data are missing.

1.244 Elementary indices are typically based on a number of series relating to diﬀerent sampled products.

The particular linked price series relating to the two qualities is therefore usually just one out of a number of parallel price series.
What may happen in practice is that the price observations for the old quality are used up to period t1 and the prices for the new quality from t onwards, the price change between t1 and t being omitted from the calculations.

In eﬀect, this amounts to using the third option: that is, estimating the missing price change on the assumption that it is equal to the average change for the other sampled products within the elementary aggregate.
1.245 It may be possible to improve on this estimate by making a careful selection of the other sampled products whose average price change is believed to be more similar to the item in question than the average for the group of sampled products as a whole.

This proce- dure is described in some detail in Chapter 7, where it is illustrated with a numerical example and described as ‘‘targeting’’ the imputation or estimation.
1.246 The general method of estimating the price on the basis of the average change for the remaining is sometimes group of products is widely used.

It described as the ‘‘overall’’ class mean method.

The more reﬁned targeted version is the ‘‘targeted’’ mean method.

In general, one or other method seems likely to be prefer- able to either of the ﬁrst two options listed above, although each case must be considered on its individual merits.
AN INTRODUCTION TO CONSUMER PRICE INDEX METHODOLOGY solution, 1.247 While the class mean method seems a sensible it may nevertheless give biased practical results, as explained in Chapter 7.

The introduction of a new quality is precisely the occasion on which a pro- ducer may choose to make a signiﬁcant price change.

Many of the most important price changes may be missed if, in eﬀect, they are assumed to be equal to the average price changes for products not subject to quality change.
1.248 It is necessary, therefore, to try to make an explicit adjustment for the change in quality, at least when a signiﬁcant quality change is believed to have occurred.

Again there are several methods that may be used.

Explicit quality adjustments 1.249 Quantity adjustments.

The quality change may take the form of a change in the physical characteristics of the product that can easily be quantiﬁed, such as change in weight, dimensions, purity, or chemical com- position of a product.
It is generally a considerable oversimpliﬁcation to assume that the quality of a pro- duct changes in proportion to the size of some single physical characteristic.

For example, most consumers are very unlikely to rate a refrigerator that has three times the capacity of a smaller one as being worth three times the price of the latter.

Nevertheless it is clearly possible to make some adjustment to the price of a new quality of diﬀerent size to make it more comparable with the price of an old quality.
There is considerable scope for the judicious, or common sense, application of relatively straightforward quality adjustments of this kind.

A thorough discussion of quality adjustments based on ‘‘size’’ is given in Chapter 7.

1.250 Diﬀerences in production or option costs.

An alternative procedure may be to try to measure the change in quality by the estimated change in the costs of producing the two qualities.
The estimates can be made in consultation with the producers of the goods or ser- vices, if appropriate.

This method, like the ﬁrst, is only likely to be satisfactory when the changes take the form of relatively simple changes in the physical character- istics of the good, such as the addition of some new feature, or option, to an automobile.

It is not satisfac- tory when a more fundamental change in the nature of the product occurs as a result of a new discovery or technological innovation.
It is clearly inapplicable, for example, when a drug is replaced by another more eﬀective variant of the same drug that also happens to cost less to produce.

1.251 Another possibility for dealing with a quality change that is more complex or subtle is to seek the advice of technical experts.
This method is especially relevant when the general consumer may not have the knowledge or expertise to be able to assess or evaluate the signiﬁcance of all of the changes that may have occurred, at least when they are ﬁrst made.

1.252 The hedonic approach.

Finally, it may be possible to systematize the approach based on produc- tion or option costs by using econometric methods to estimate the impact of observed changes in the char- acteristics of a product on its price.
In this approach, the market prices of a set of diﬀerent qualities or models are regressed on what are considered to be the most important physical or economic characteristics of the diﬀerent models.

This approach to the evaluation of quality change is known as hedonic analysis.

When the characteristics are attributes that cannot be quantiﬁed, they are represented by dummy variables.
The regres- sion coeﬃcients measure the estimated marginal eﬀects of the various characteristics on the prices of the models and can therefore be used to evaluate the eﬀects of changes in those characteristics, i.e., changes in quality, over time.

1.253 The hedonic approach to quality adjustment can provide a powerful, objective and scientiﬁc method of evaluating changes in quality for certain kinds of products.

It has been particularly successful in dealing with computers.
The economic theory underlying the hedonic approach is examined in more detail in Chapter 21.

The application of the method is explained in some detail in Chapter 7.

Products can be viewed as bundles of characteristics that are not individually priced, as the consumer buys the bundle as a single package.

The objective is to try to ‘‘unbundle’’ the characteristics to estimate how much they contribute to the total price.
In the case of computers, for example, three basic char- acteristics are the processor speed, the size of the RAM and the hard drive capacity.

An example of a hedonic re- gression using these characteristics is given in Chapter 7.

1.254 The results obtained by applying hedonics to computer prices have had a considerable impact on attitudes towards the treatment of quality change in CPIs.
They have demonstrated that for goods where there are rapid technological changes and improvements in quality, the size of the adjustments made to the market prices of the products to oﬀset the changes in the quality can largely determine the movements of the elementary price index.

For this reason, the manual contains a thorough treatment of the use of hedonics.
Chapter 7 provides further analysis, including a comparison show- ing that the results obtained by using hedonics and matched models can diﬀer signiﬁcantly when there is a high turnover of models.

1.255 It may be concluded that statistical oﬃces must pay close attention to the treatment of quality change and try to make explicit adjustments whenever possible.

The importance of this topic can scarcely be over-emphasized.
The need to recognize and adjust for changes in quality has to be impressed on price collec- tors.

Failure to pay proper attention to quality changes can introduce serious biases into a CPI.

Item substitution and new goods 1.256 As noted above, ideally price indices would seek to measure pure price changes between matched products that are identical in the two periods compared.
However, as explained in Chapter 8, the universe of products that a CPI has to cover is a dynamic universe that is gradually changing over time.

Pricing matched products constrains the selection of products to a static universe of products given by the intersection of the two sets of products existing in the two periods compared.
29 CONSUMER PRICE INDEX MANUAL: THEORY AND PRACTICE This static universe, by deﬁnition, excludes both new products and disappearing products, whose price behav- iour is likely to diverge from that of the matched prod- ucts.

Price indices have to try to take account of the price behaviour of new and disappearing products as far as possible.

1.257 A formal consideration and analysis of these problems are given in Appendix 8.1 to Chapter 8.
A replacement universe is deﬁned as one that starts with the base period universe but allows new products to enter as replacements as some products disappear.

Of course, quality adjustments of the kind discussed above are needed when comparing the prices of the replacement products with those of the products that they replace.

1.258 One way in which to address the underlying problem of the changing universe is by sample rotation.
This requires a completely new sample of products to be drawn to replace the existing one.

The two samples must overlap in one period that acts as the link period.

This procedure can be viewed as a systematic exploitation of the overlap method of adjusting for quality change.
It may not therefore deal satisfactorily with all changes in quality that occur, because the relative prices of diﬀerent goods and services at a single point of time may not provide satisfactory measures of the relative qualities of all the goods and services concerned.

Nevertheless, fre- quent sample rotation helps by keeping the sample up to date and may reduce the extent to which explicit quality adjustments are required.

Sample rotation is expensive, however.


New goods and services 1.259 The diﬀerence in quality between the original product and the one that it replaces may become so great that the new quality is better treated as a new good, although the distinction between a new quality and a new good is inevitably somewhat arbitrary.

As noted in Chapter 8, a distinction is also drawn in the economics literature between evolutionary and revolu- tionary new goods.
An evolutionary new good or service is one that meets existing needs in much more eﬃcient or new ways, whereas a revolutionary new good or service provides completely new kinds of services or beneﬁts.

In practice, an evolutionary new good can be ﬁtted into some sub-class of the product or expenditure classiﬁca- tion, whereas a revolutionary new good will require some modiﬁcation to the classiﬁcation in order to accommodate it.

1.260 There are two main concerns with new goods or services.
The ﬁrst relates to the timing of the intro- duction of the new product into the index.

The second relates to the fact that the mere availability of the new product on the market may bring a welfare gain to consumers, whatever the price at which it is sold initial- ly.

Consider, for example, the introduction of the ﬁrst antibiotic drug, penicillin.

The drug provided cures for conditions that previously might have been fatal.

The beneﬁt might be virtually priceless to some individuals.
One way of gauging how much beneﬁt is gained by the introduction of a new good is to ask how high its price would have to be to reduce the demand for the product 30 to zero.

Such a price is called the ‘‘demand reservation price’’.

It could be very high indeed in the case of a new life-saving drug.

If the demand reservation price could be estimated, it could be treated as the price in the period just before the new product appeared.
The fall between the demand reservation price and the price at which the product actually makes its ﬁrst appearance could be included in the CPI.

1.261 In practice, of course, statistical oﬃces cannot be expected to estimate demand reservation prices with suﬃcient reliability for them to be included in a CPI.
The concept is nevertheless useful because it highlights the fact that the mere introduction of a new good may bring a signiﬁcant welfare gain that could be reﬂected in the CPI, especially if it is intended to be a COLI.

In general, any enlargement of the set of consumption possibilities open to consumers has the potential to make them better oﬀ, other things being equal.
1.262 It is often the case that new goods enter the market at a higher price than can be sustained in the longer term, so their prices typically tend to fall relatively over the course of time.

Conversely, the quantities pur- chased may be very small initially but increase sig- niﬁcantly.

These complications make the treatment of new products particularly diﬃcult, especially if they are revolutionary new goods.
Because of both the welfare gain from the introduction of a new product and the tendency for the price of a new good to fall after it has been introduced, it is possible that important price re- ductions may fail to be captured by CPIs because of the technical diﬃculties created by new products.

Chapter 8 concludes by expressing concern about the capacity of CPIs to deal satisfactorily with the dynamics of modern markets.
In any case, it is essential that statistical oﬃces are alert to these issues and adopt procedures that take account of them to the maximum extent possible, given the data and resources available to them.

Calculation of consumer price indices in practice 1.263 Chapter 9 provides a general overview of the ways in which CPIs are calculated in practice.

The methods used in diﬀerent countries are by no means all the same, but they have much in common.
There is from users as well as compilers in clearly interest knowing how most statistical oﬃces set about calculat- ing their CPIs.

The various stages in the calculation process are illustrated by numerical examples.

The chapter is descriptive and not prescriptive, although it does try to evaluate the strengths and weaknesses of existing methods.
It makes the point that because of the greater insights into the properties and behaviour of indices gained in recent years, it is now recognized that not all existing practices are necessarily optimal.

1.264 As the various stages involved in the calcula- tion process have, in eﬀect, already been summarized in the preceding sections of this chapter, it is not proposed to repeat them all again in this section.

It may be useful, however, to give an indication of the nature of the con- tents of Chapter 9.
AN INTRODUCTION TO CONSUMER PRICE INDEX METHODOLOGY Elementary price indices 1.265 Chapter 9 starts by describing how the ele- mentary aggregates are constructed by working down from groups, classes and sub-classes of COICOP, or some equivalent expenditure classiﬁcation.

It reviews the principles underlying the delineation of the elementary aggregates themselves.
Elementary aggregates are in- tended to be as homogeneous as possible, not merely in terms of the physical and economic characteristics of the their price products covered but also in terms of movements.

1.266 Chapter 9 then considers the consequences of using alternative elementary index formulae to calculate the elementary indices.

It proceeds by means of a series of numerical examples that use simulated price data for four diﬀerent products within an elementary aggregate.
The elementary indices themselves, and their properties, have already been explained above.

An elementary price index may be calculated either as a chain index or as a direct index; that is, either by comparing the price each month, or quarter, with that in the immediately pre- ceding period or with the price in the ﬁxed price refer- ence period.

Table 9.1 of Chapter 9 uses both approa- ches to illustrate the calculation of three basic types of elementary index, Carli, Dutot and Jevons.
It is designed to highlight a number of their properties.

For example, it shows the eﬀects of ‘‘price bouncing’’ in which the same four prices are recorded for two consecutive months, but the prices are switched between the four products.

The Dutot and Jevons indices record no increase but the Carli index registers an increase.

Table 9.1 also illustrates the diﬀerences between the direct and the chain indices.

After six months, each of the four prices is 10 per cent higher than at the start.
Each of the three direct indices records a 10 per cent increase, as also do the chained Dutot and Jevons indices because they are transitive.

The chained Carli, however, records an increase of 29 per cent, which is interpreted as illus- trating the systematic upward bias in the Carli formula resulting from its failure to satisfy the time reversal test.
It is noted in Chapter 9 that the chaining and direct approaches have diﬀerent implications when there are missing price observations, quality changes and re- placements.

The conclusion is that the use of a chain index can make the estimation of missing prices and the introduction of replacement items easier from a com- putational point of view.
1.267 1.268 Chapter 9 also examines the eﬀects of missing price observations, distinguishing between those that are temporarily missing and those that have become per- manently unavailable.

Table 9.2 contains a numerical example of the treatment of the temporarily missing prices.

One possibility is simply to omit the product whose price is missing for one month from the calcula- tion of indices that compare that month with the pre- ceding and following months, and also with the base period.
Another possibility is to impute a price change on the basis of the average price for the remaining products, using one or other of the three types of average.

The example is a simpliﬁed version of the kind of examples that are used in Chapter 7 to deal with the same problem.

1.269 Tables 9.3 and 9.4 illustrate the case in which one product disappears permanently to be replaced by another product.
In Table 9.3 there is no overlap between the two products and the options considered are again to omit the products or to impute price changes for them based on averages for the other prod- ucts.

Table 9.4 illustrates the situation in which the products overlap in one month.
1.270 Chapter 9 also considers the possibility that there may be some expenditure weights available within an elementary aggregate, in which case it may be pos- sible to calculate a Laspeyres or a geometric Laspeyres index, these being the weighted versions of the Carli and the Jevons.

Higher-level indices 1.271 Later sections of Chapter 9 illustrate the cal- culation of the higher-level indices using the elementary price indices and the weights provided by the elementary expenditure aggregates.
It is at this stage that the tra- index number theory that was summarized ditional earlier in this chapter and is explained in detail in Chapters 15 to 19 comes into play.

1.272 At the time the monthly CPI is ﬁrst calculated, the only expenditure weights available must inevitably refer to some earlier period or periods of time.
As explained earlier in this chapter, this predisposes the CPI to some form of Lowe or Young index in which the quantities, or expenditures, refer to some weight refer- ence period b which precedes the price reference period 0.

Such indices are often loosely described as Laspeyres type indices, but this description is inappropriate.
At some later date, however, estimates may become avail- able of the expenditures in both the price reference period 0 and the current period t, so that retrospectively the number of options open is greatly increased.

It then becomes possible to calculate both Laspeyres and Paasche type indices, and also superlative indices such as Fisher or To¨ rnqvist.

There is some interest in calculating such indices later, if only to see how the original indices compare with the superlative indices.
Some countries may wish to calculate retrospective superlative indices for that reason.

Although most of the discussion in Chapter 9 focuses on some type of Lowe index because the oﬃcial index ﬁrst published will inevitably be of that type, this should not be interpreted as implying that such an index is the only possibility in the longer term.

1.273 Production and maintenance of higher-level indices.
In practice, the higher-level indices up to and including the overall CPI are usually calculated as Young indices; that is, as weighted averages of the ele- mentary price indices using weights derived from expen- ditures in some earlier weight reference period.

This is a relatively straightforward operation, and a numerical example is given in Table 9.5 of Chapter 9 in which, for simplicity, the weight and price reference periods are assumed to be the same.
Table 9.6 illustrates the case in which weight and price reference periods are not the same, and the weights are price updated between weight reference period b and the price reference period 0.
It illustrates the point that statistical oﬃces have two 31 CONSUMER PRICE INDEX MANUAL: THEORY AND PRACTICE options when a new price reference period is introduced: they can either preserve the relative quantities of the weight reference period or they can preserve the relative expenditures, but they cannot do both.

Price updating preserves the quantities.

1.274 The introduction of new weights is a necessary and integral part of the compilation of a CPI over the long run.
Weights have to be updated sooner or later, some countries preferring to update their weights each year.

Whenever the weights are changed, the index based on the new weights has to be linked to the index based on the old weights.

Thus, the CPI inevitably becomes a chain index over the long term.

An example of the linking is given in Table 9.7.


Apart from the technicalities of the linking process, the introduction of new weights, especially if carried out at intervals of ﬁve years or so, provides an opportunity to undertake a major review of the whole methodology.

New products may be introduced into the index, classiﬁcations may be revised and updated, while even the index number for- mula might be changed.
Annual chaining facilitates the introduction of new products and other changes on a more regular basis, but in any case some ongoing is maintenance of annually chained or not.

the index is needed whether it 1.275 Chapter 9 concludes with a section on data editing, a process that is very closely linked to the actual calculation of the elementary prices indices.

Data editing comprises two steps: the detection of possible errors and outliers, and the verifying and correction of the data.
Eﬀective monitoring and quality control are needed to ensure the reliability of the basic price data fed into the calculation of the elementary prices indices, on which the quality of the overall index depends.

Organization and management 1.276 The collection of price data is a complex operation involving extensive ﬁeldwork by a large num- ber of individual collectors.
The whole process requires careful planning and management to ensure that data collected conform to the requirements laid down by the central oﬃce with overall responsibility for the CPI.

Appropriate management procedures are described in Chapter 12 of this manual.

1.277 Price collectors should be well trained to ensure that they understand the importance of selecting the right products for pricing.

Inevitably, price collectors are bound to use their own discretion to a considerable extent.
As already explained, one issue of crucial importance to the quality and reliability of a CPI is how to deal with the slowly evolving set of products with which a price collector is confronted.

Products may disappear and have to be replaced by others, but it may also be appropriate to drop some products before they disappear altogether, they have become unrepre- sentative.

Price collectors need to be provided with appropriate training and very clear instructions and documentation about how to proceed.
Clear instructions if 32 are also needed to ensure that price collectors collect the right prices when there are sales, special oﬀers or other exceptional circumstances.

1.278 As just noted, the price data collected have also to be subjected to careful checking and editing.

Many checks can be carried out by computer, using standard statistical control methods.

It may also be useful to send out auditors to accompany price collectors and monitor their work.
The various possible checks and controls are explained in detail in Chapter 12.

1.279 Improvements in information technology should obviously be exploited to the fullest extent pos- sible.

For example, collectors may use hand-held com- puters and transmit their results electronically to the central oﬃce.
Publication and dissemination 1.280 As noted above and in Chapter 2, the CPI is an extremely important statistic whose movements can inﬂuence the central bank’s monetary policy, aﬀect stock markets, inﬂuence wage rates and social security payments, and so on.

There must be public conﬁdence in its reliability, and in the competence and integrity of those responsible for its compilation.

The methods used to compile it must therefore be fully documented, trans- parent and open to public scrutiny.
Many countries have an oﬃcial CPI advisory group consisting of both experts and users.

The role of such a group is not just to advise the statistical oﬃce on technical matters but also to promote public conﬁdence in the index.

1.281 Users of the index also attach great impor- tance to having the index published as soon as possible after the end of each month or quarter, preferably within two or three weeks.

There are also many users who do not wish the index to be revised once it has been published.
Thus there is likely to be some trade-oﬀ between the timeliness and the quality of the index.

1.282 Publication should be understood to mean the dissemination of the results in any form.

In addition to publication in print, or hard copy, the results should be released electronically and be available through the Internet on the web site of the statistical oﬃce.

1.283 As explained in Chapter 13, good publication policy goes beyond timeliness, conﬁdence and transpar- ency.
The results should be made available to all users, in both the public and the private sectors, at the same time and according to a publication schedule announced in advance.

There should be no discrimination among users in the timing of the release of the results.

The results should not be subject to governmental scrutiny as a condition for their release, and should be seen to be free from political or other pressures.
1.284 There are many decisions to be taken about the degree of detail in the published data and the dif- ferent ways in which the results may be presented.

Users need to be consulted about these questions.

These issues are discussed in Chapter 13.

As they do not aﬀect the actual calculation of the index, they need not be pursued further at this point.

USES OF CONSUMER PRICE INDICES 2 2.1 The consumer price index (CPI) is treated as a key indicator of economic performance in most coun- tries.
The purpose of this chapter is to explain why CPIs are compiled and what they are used for.

A range of possible consumer price indices 2.2 As noted in Chapter 1, compilers have to take into account the needs of users in deciding on the group of households and range of consumption goods and services covered by a CPI.
As the prices of diﬀerent goods and services do not all change at the same rate, or even all move in the same direction, changing the cov- erage of the index will change the value of the index.

Thus, there can be no unique CPI and a range of pos- sible CPIs could be deﬁned.
2.3 While there may be interest in a CPI which is as broadly deﬁned as possible, covering all the goods and services consumed by all households, there are many other options for deﬁning CPIs covering particular sets of goods and services, which may be more useful for particular analytic or policy purposes.

There is no ne- cessity to have only a single CPI.

When only a single CPI is compiled and published, there is a risk that it may be used for purposes for which it is not appropriate.
More than one CPI could be published in order to meet dif- ferent analytic or policy needs.

It is important to recog- nize, however, that the publication of more than one CPI can be confusing to users who view consumer inﬂation as a pervasive phenomenon aﬀecting all households equally.

The coexistence of alternative measures could undermine their credibility for many users.
2.4 This chapter is intended not only to describe the most important uses for CPIs, but also to indicate how the coverage of a CPI can be aﬀected by the use for which it is intended.

The question of what is the most appropriate coverage of a CPI must be addressed before considering what is the most appropriate methodology to be used.
Whether or not the CPI is intended to be a it is still necessary to cost of living index (COLI), determine exactly what kinds of good and services and what types of households are meant to be covered.

This can only be decided on the basis of the main uses of the index.

Indexation 2.5 Indexation is a procedure whereby the monetary values of certain payments, or stocks, are increased or decreased in proportion to the change in the value of some price index.
Indexation is most commonly applied to monetary ﬂows such as wages, rents, interest or taxes, but it may also be applied to the capital values of certain monetary assets and liabilities.

Under conditions of high inﬂation, the use of indexation may become widespread throughout the economy.
2.6 The objective of indexation of money incomes may be either to maintain the purchasing power of those incomes in respect of certain kinds of goods and ser- vices, or to preserve the standard of living or welfare of the recipients of the incomes.

These two objectives are not quite the same, especially over the longer term.
Maintaining purchasing power may be interpreted as changing money income in proportion to the change in the monetary value of a ﬁxed basket of goods and ser- vices purchased out of that income.

As explained further below and in detail in Chapter 3, maintaining the pur- chasing power of income over a ﬁxed set of goods and services does not imply that the standard of living of the recipients of the income is necessarily unchanged.
2.7 When the indexation applies to monetary assets or liabilities, it may be designed to preserve the real value of the asset or liability relative to other assets or relative to the values of speciﬁed ﬂows of goods and services.

Indexation of wages 2.8 As noted in Chapters 1 and 15, the indexation of wages seems to have been the main objective behind the original compilation of CPIs as the practice goes back over two centuries, although there has always been general interest in measuring inﬂation.
If the indexation of wages is the main justiﬁcation for a CPI, it has direct implications for the coverage of the index.

First, it sug- gests that the index should be conﬁned to expenditures made by households whose principal source of income is wages.

Second, it may suggest excluding expenditures on certain types of goods and services which are considered to be luxurious or frivolous.

If so, value judgements or political judgements may enter into the selection of goods and services covered.
This point is elaborated further below.

Indexation of social security beneﬁts 2.9 It has become common practice in many coun- tries to index-link the rates at which social security beneﬁts are payable.

There are many kinds of beneﬁts, such as retirement pensions, unemployment beneﬁts, sickness beneﬁts, child allowances, and so on.
As in the case of wages, when index-linking to beneﬁts of this kind is the main reason for compiling the CPI, it may suggest 33 CONSUMER PRICE INDEX MANUAL: THEORY AND PRACTICE restricting the coverage of the index to certain types of households and goods and services.

Many kinds of goods and services may then be excluded by political decision on the grounds that they are unnecessary or inappro- priate.
This type of thinking may lead to pressure to exclude expenditures on items such as holidays, gam- bling, tobacco or alcoholic drink.

2.10 An alternative procedure is to compile separate CPIs for diﬀerent categories of households.

For exam- ple, an index may be compiled covering the basket of goods and services purchased by households whose principal source of income is a social security pension.
When this is done, it may be superﬂuous to decide to exclude certain types of luxury or inappropriate expen- ditures, as the actual expenditures on such items may be negligible anyway.

2.11 As already noted, publishing more than one CPI may be confusing if inﬂation is viewed as aﬀecting everyone in the same way.

Such confusion can be avoi- ded by suitable publicity; it is not diﬃcult to explain the fact that price changes are not the same for diﬀerent categories of expenditures.
In practice, some countries do publish more than one index.

2.12 The main reason why it may not be justiﬁable to publish more than one index is that the movements in the diﬀerent indices may be virtually the same, especially in the short term.

In such cases, the costs of compiling and publishing separate indices may not be worthwhile.
it may need much bigger diﬀerences in In practice, patterns of expenditure than actually exist between dif- ferent groups of households to yield signiﬁcantly dif- ferent CPIs.
2.13 Finally, it should be noted that the deliberate exclusion of certain types of goods and services by po- litical decision on the grounds that the households towards whom the index is targeted ought not to be purchasing such goods, or ought not to be compensated for increases in the prices of such goods, cannot be recommended because it exposes the index to political manipulation.

For example, suppose it is decided that certain products such as tobacco or alcoholic drink should be excluded from a CPI.
There is then a possibility that when taxes on products have to be increased, these products may be deliberately selected in the knowledge that the resulting price increases do not increase the CPI.

Such practices are not unknown.


The type of index used for indexation 2.14 When income ﬂows such as wages or social security beneﬁts are index-linked, it is necessary to consider the implications of choosing between a cost of living index and a price index that measures the changes in the cost of purchasing a ﬁxed basket of goods and services, a type of index described here as a Lowe index.

The widely used Laspeyres and Paasche indices are examples of Lowe indices.
The Laspeyres index uses a typical basket purchased in the earlier of the two periods compared, while the Paasche uses a basket typical of the later period.

This ‘‘ﬁxed basket’’ method has a long history, as explained in Chapter 15.

In contrast, a cost of living index (COLI) compares the cost of two baskets 34 that may not be exactly the same but which bring the same satisfaction or utility to the consumer.
2.15 Indexation using a Laspeyres price index will tend to over-compensate the income recipients for changes in their cost of living.

Increasing incomes in proportion to the change in the cost of purchasing a basket purchased in the past ensures that the income recipients have the opportunity to continue purchasing that same basket if they wish to do so.

They would then be at least as well oﬀ as before.


However, by adjusting their pattern of expenditures to take account of changes in the relative prices of the goods and services they buy, they will be able to improve their standard of living or welfare because they can substitute goods that have become relatively cheaper for ones that have become relatively dearer.
In addition, they may be able to start to purchase completely new kinds of goods which pro- vide new kinds of beneﬁts that were not available in the earlier period.

Such new goods tend to lower a cost of living index when they ﬁrst appear even though no price can actually be observed to fall, as there was no previous price.

Indexation of interest, rents and other contractual payments 2.16 It is common for payments of both rents and interest to be index-linked.
Governments may issue bonds with an interest rate speciﬁcally linked to the CPI.

The interest payable in any given period may be equal to a ﬁxed real rate of interest plus the percentage increase in the CPI.

Payments of housing rents may also be linked to the CPI or possibly to some other index, such as an index of house prices.

2.17 Creditors receiving interest payments do not consist only of households, of course.
In any case, the purpose of index-linking interest is not to maintain the standard of living of the creditors but rather to maintain their real wealth by compensating them for the real hold- ing, or capital, losses on their loans incurred as a result of general inﬂation.

A CPI may not be the ideal index for this purpose but may be used by default in the absence of any other convenient index, a point discussed further below.

2.18 Many other forms of contractual payments may be linked to the CPI.
For example, legal obligations to pay alimony or for the support of children may be linked to the CPI.

Payments of insurance premiums may be linked either to the index as a whole or to a sub-index relating to some speciﬁc types of expenditures, such the costs of repairs.

Taxation 2.19 Movements in a CPI may be used to aﬀect the amounts payable in taxation in several ways.
For exam- ple, liability for income tax may be aﬀected by linking personal allowances that are deductible from taxable income to changes in the CPI.

Under a system of pro- gressive taxation, the various thresholds at which higher rates of personal income tax become operative may be changed in proportion to changes in the CPI.
Liability for capital gains tax may be reduced by basing it on real rather than nominal capital gains through reducing the percentage increase in the value of the asset by the per- centage change in the CPI over the same period, for taxation purposes.

In general, there are various ways in which some form of indexation may be introduced into tax legislation.
Real consumption and real income 2.20 Price indices can be used to deﬂate expenditures at current prices or money incomes in order to derive measures of real consumption and real income.

Real measures involve volume comparisons over time (or space).

There are two diﬀerent approaches to such com- parisons which are analogous to the distinction between a Lowe, or basket, index and a cost of living index.
2.21 The ﬁrst deﬁnes the change in real consumption as the change in the total value of the goods and services actually consumed measured at the ﬁxed prices of some chosen period.

This is equivalent to deﬂating the change in the current value of the goods and services consumed by an appropriately weighted Lowe price index.

The change in real income can be measured by deﬂating the change in total money income by the same price index.
2.22 The alternative approach deﬁnes the change in real consumption as the change in welfare derived from the goods and services actually consumed.

This may be estimated by deﬂating the change in the current value of consumption by using a COLI.

Real income may be similarly obtained by deﬂating money income by the same COLI.

2.23 The two approaches cannot lead to the same results if the pure price index and the COLI diverge.
The choice between the two approaches to the measurement of real consumption and real income will not pursued further here, as the issues involved are essentially the same as those already considered above in the parallel discussion of the choice between a Lowe, or basket, price index and a cost of living index.
Consistency between price indices and expenditure series 2.24 The data collected on prices and the data col- lected on household expenditures must be mutually con- sistent when measuring real consumption.

This requires that both sets of data should cover the same set of goods and services and use the same concepts and classiﬁca- tions.
Problems may arise in practice because the price indices and the expenditure series are often compiled independently of each other by diﬀerent departments of a statistical agency or even by diﬀerent agencies.

2.25 The coverage of a CPI need not be the same as that of total household consumption expenditures in the national accounts.

The CPI may be targeted at selected households and expenditures for reasons given above.
However, the diﬀerence in coverage between the CPI and the national accounts expenditures must be pre- cisely identiﬁed so that it is possible to account for the diﬀerences between them.

The price index used to deﬂate the expenditures ought to cover the additional goods and services not covered by the CPI.

This may not be USES OF CONSUMER PRICE INDICES easy to achieve in practice because the relevant price data may not be easily available if the price collection procedures are geared to the CPI.
Moreover, even if all the basic price data are available, the price index needed for deﬂation purposes is likely to be of a diﬀerent type or formula from the CPI itself.

2.26 In principle, the deﬂation of national accounts estimates will normally require the compilation of appropriately deﬁned price indices that diﬀer from the CPI but may draw on the same price database.
They may diﬀer from the CPI not only in the range of the price and expenditure data they cover and the weighting and index number formula employed, but also in the frequency with which they are compiled and the length of the time periods they cover.

The movements of the resulting indices will tend to diﬀer somewhat from the CPI pre- cisely because they measure diﬀerent things.
Although designed to be used to deﬂate expenditure data, they also provide useful additional information about movements in consumer prices.

This information complements and supplements that provided by the CPI.

The CPI itself is not designed to serve as a deﬂator.

Its coverage and methodology should be designed to meet the needs of the CPI as described in other sections of this chapter.
2.27 When other types of consumer price indices are needed in addition to the CPI, this should be recognized at the data collection stage as it may be more eﬃcient and cost eﬀective to use a single collection process to meet the needs of more than one kind of price index.

This may imply collecting rather more price data than are needed for the CPI itself if the coverage of the CPI has been deliberately restricted in some way.
Purchasing power parities 2.28 Many countries throughout the world, includ- ing all the member countries of the European Union (EU), participate in regular international programmes enabling purchasing power parities (PPPs) to be calcu- lated for household consumption expenditures.

The calculation of PPPs requires the prices of individual con- sumer goods and services to be compared directly be- tween diﬀerent countries.
In eﬀect, PPP programmes involve the compilation of international consumer price indices.

Real expenditures and real incomes can then be compared between countries in much the same way as between diﬀerent time periods in the same country.

2.29 It is not proposed to examine PPP methodol- ogy here but simply to note that PPPs create yet another demand for basic price data.

When such data are being collected, therefore, it is important to recognize that they can be used for PPPs as well as CPIs.
PPPs are essentially international deﬂators which are analogous to the inter-temporal deﬂators needed for the national accounts of a single country.

Thus, while the processing and aggregation of the basic data for CPI purposes should be determined by the needs of the CPI itself, it is appropriate to take account of the requirements of these other kinds of price indices at the data collection stage.
There may be important economies of scale to be re- alized by using a single collection process to meet the needs of several diﬀerent types of indices.

35 CONSUMER PRICE INDEX MANUAL: THEORY AND PRACTICE 2.30 Thus, operationally as well as conceptually, the CPI needs to be placed in the context of a wider set of interrelated price indices.

The compilation of CPIs pre- dates the compilation of national accounts by many years in some countries, so the CPI may have originated as a free-standing index.
The CPI can, however, no longer be treated as an isolated index whose compilation and methodology can proceed quite independently of other interrelated statistics.

Use of the consumer price index for accounting under inﬂation 2.31 When there is inﬂation, both business and national accounts have to introduce adjustments which are not needed when the price level is stable.

This is a complex subject which cannot be pursued in any depth here.
Two methods of accounting are commonly used, and they are summarized below.

Both require price indices for their implementation.

power accounts 2.32 Current Current purchasing power accounts purchasing are accounts in which the monetary values of the ﬂows in earlier time periods are scaled up in proportion to the increase in some general index of inﬂation between the earlier period and the current period.
In principle, the index used should be a general price index covering other ﬂows in addition to household consumption expenditures, but in practice the CPI is often used by default in the absence of a suitable general index.
Current cost accounting 2.33 Current cost accounting is a method of accounting for the use of assets in which the cost of using the assets in production is calculated at the current prices of those assets as distinct from the prices at which the assets were purchased or otherwise acquired in the past (historic costs).

The current cost of using an asset takes account not only of changes in the general price level but also of changes in the relative price of that type of asset since it was acquired.
In principle, the price indices that are used to adjust the original prices paid for the assets should be speciﬁc price indices relating to that particular type of asset, and such indices are calculated and used in this way in some countries.

However, when there are no such indices available there remains the possibility of using the CPI, or some sub-index of the CPI, by default, and CPIs have been used for this purpose.
tion is needed in order to set targets and also to judge the degree of success achieved by the government or central bank in meeting anti-inﬂationary targets.

 As noted above, a measure of general inﬂation is also needed for both business and national accounting purposes, particularly for current purchasing power accounting.

 The concept of a relative price change is important in economics.
It is convenient therefore to be able to measure the actual changes in the prices of individual goods or services relative to some measure of general inﬂation.

There is also a need to be able to measure real holding (or capital) gains and losses on assets, including monetary assets and liabilities.

2.35 Suitable measures of general inﬂation are con- sidered in Chapter 14, in which it shown that a hierarchy of price indices exists that includes the CPI.
Clearly, a CPI is not a measure of general inﬂation, as it only measures changes in the prices of consumer goods and services purchased by households.

A CPI does not cover capital goods, such as houses, or the goods and services consumed by enterprises or the government.

Any attempt to analyse inﬂationary pressures in the economy must also take account of other price movements, such as changes in the prices of imports and exports, the prices of industrial inputs and outputs, and also asset prices.
Consumer price indices and inﬂation targets 2.36 Despite the obvious limitations of a CPI as a measure of general inﬂation, it is commonly used by governments and central banks to set inﬂation targets.

Similarly, it is interpreted by the press and the public as the ultimate measure of inﬂation.
Although govern- ments and central banks are obviously well aware of the fact that the CPI is not a measure of general inﬂation, a number of factors help to explain the popularity of the CPI, and these are discussed below.

2.37 It may be noted, however, that even though the CPI does not measure general inﬂation its movements may be expected to be highly correlated with those of a more general measure, if only because consumption expenditures account for a large proportion of total ﬁnal expenditures.
In particular, the CPI should provide a reliable indicator of whether inﬂation is accelerating or decelerating and also of any turning points in the rate of inﬂation.

This information is highly valuable even if the CPI may be systematically understating or overstating the general rate of inﬂation.
Consumer price indices and general inﬂation 2.34 As already noted, measures of the general rate of inﬂation in the economy as a whole are needed for various purposes: Controlling inﬂation is usually one of the main objectives of government economic policy, although responsibility for controlling inﬂation may be dele- gated to the central bank.
A measure of general inﬂa- Consumer price indices and international comparisons of inﬂation 2.38 CPIs are also commonly used to make inter- national comparisons of inﬂation rates.

An important example of their use for this purpose is provided by the EU.
In order to judge the extent to which rates of inﬂation in the diﬀerent member countries were conver- ging in the mid-1990s prior to the formation of the the member countries European Monetary Union, 36 decided in the Maastricht Treaty that CPIs should be used.

Although CPIs measure consumer inﬂation rather than general inﬂation, their use to evaluate the extent of convergence of inﬂation may be justiﬁed on similar grounds to those just mentioned.
Presumably, the con- vergence in CPIs will be highly correlated with that in general inﬂation, so the use of a speciﬁc rather than a general measure of inﬂation may lead to the same con- clusions about the extent of convergence and which countries diverge the most from the average.

Popularity of consumer price indices as economic statistics 2.39 CPIs seem to have acquired a unique status among economic statistics in most countries.
There are several factors which help to explain this: First, all households have their own personal experi- ence of the phenomenon the CPI is supposed to be measuring.

The general public are very conscious of changes in the prices of consumer goods and services, and the direct impact those changes have on their living standards.

Interest in CPIs is not conﬁned to the press and politicians.

 Changes in the CPI tend to receive a lot of publicity.

Their publication can make headline news.


The CPI is a high-proﬁle statistic.

 The CPI is published frequently, usually each month, so that the rate of consumer inﬂation can be closely monitored.

The CPI is also a timely statistic that is released very soon after the end of the period to which it refers.

 The CPI is a statistic with a long history, as noted in Chapters 1 and 15.

People have been familiar with it for a long time.


 Although price changes for certain kinds of consumer goods are diﬃcult to measure because of quality changes, price changes for other kinds of goods and services such as capital goods and government ser- vices, especially public services, tend to be even more diﬃcult to measure.

The CPI may be a relatively reliable price index compared with the price indices for some other ﬂows.

 The CPI is widely respected.


Its accuracy and reli- ability are seldom seriously questioned.

 Most countries have deliberately adopted a policy of not revising the index once it has been published.

This makes it more attractive for many purposes, especially those with ﬁnancial consequences such as indexation.

The lack of revisions may perhaps create a somewhat spurious impression of certainty, but it also seems to enhance the credibility and acceptability of the index.
USES OF CONSUMER PRICE INDICES 2.40 The widespread use of the CPI for more pur- poses than it is designed for can be explained by the various factors listed above, together with the fact that no satisfactory alternative or more comprehensive mea- sures of inﬂation are available monthly in most coun- tries.
For example, the CPI may be used as a proxy for a more general measure of inﬂation in business account- ing, even though it may be clear that, conceptually, the CPI is not the ideal index for the purpose.

Similarly, the fact that the CPI is not subject to revision, together with its frequency and timeliness, may explain its popularity for indexation purposes in business or legal contracts in contexts where it also may not be very appropriate conceptually.
These practices may be defended on the grounds that the alternative to using the CPI may be to make no adjustment for inﬂation.

Although the CPI may not be the ideal measure, it is much better to use it than to make no adjustment whatsoever.

2.41 Although the CPI is often used as a proxy for a general measure of inﬂation, this does not justify extend- ing its coverage to include elements that go beyond household consumption.
If broader indices of inﬂation are needed, they should be developed in addition to the CPI, leaving the CPI itself intact.

Some countries are in fact developing additional and more comprehensive measures of inﬂation within the kind of conceptual framework outlined in Chapter 14 below.

The need for independence and integrity of consumer price indices 2.42 Because of the widespread use of CPIs for all kinds of indexation, movements in the CPI can have major ﬁnancial ramiﬁcations throughout the economy.
The implications for the government alone can be con- siderable, given that the CPI can aﬀect interest pay- ments and taxation receipts as well as the government’s wage and social security outlays.

2.43 When ﬁnancial interests are involved, there is always a risk that both political and non-political pres- sure groups may try to exert an inﬂuence on the meth- odology used to compile the CPI.

The CPI, in common with other oﬃcial statistics, must be protected from such pressures and be seen to be protected.
Partly for this reason, many countries establish an advisory committee to ensure that the CPI is not subject to outside inﬂuence.

The advisory committee may include representatives of a cross-section of interested parties as well as indepen- dent experts able to oﬀer professional advice.

Informa- tion about the methodology used to calculate CPIs should be publicly available.
37 CONCEPTS AND SCOPE Introduction 3.1 The purpose of this chapter is to deﬁne and clarify the basic concepts of price and consumption used in a consumer price index (CPI) and to explain the scope of the index.

While the general purpose of a consumer price index is to measure changes in the prices of con- sumption goods and services, the concept of ‘‘consump- tion’’ is an imprecise one that can be interpreted in several diﬀerent ways, each of which may lead to a dif- ferent CPI.
The governmental agency or statistical oﬃce responsible for compiling a CPI also has to decide whe- ther the index is meant to cover all consumers, i.e., all households, or to be restricted to a particular group of households.

The precise scope of a CPI is inevitably inﬂuenced by what is intended, or believed, to be the main use of the index.
Statistical oﬃces should, however, bear in mind that CPIs are widely used as measures of general inﬂation, even though they may not have been designed for this purpose.

3.2 Consumption is an activity in which persons, acting either individually or collectively, use goods or services to satisfy their needs and wants.

In economics, no attempt is made to observe and record such activities directly.
Instead, consumption is measured either by the value of the goods and services wholly or partly used up in some period, or by the value of the goods and services that are purchased, or otherwise acquired, for purposes of consumption.

3.3 A consumer price index can have two diﬀerent meanings, as ‘‘consumer’’ may refer either to a type of economic unit, typically a person or a household, or to a certain type of good or service.
To avoid confusion, the term ‘‘consumer’’ will, so far as possible, be reserved here for persons or households, while so-called ‘‘consumer’’ goods will be described as ‘‘consumption’’ goods.

A con- sumption good or service is deﬁned as one that members of households use, directly or indirectly, to satisfy their own personal needs and wants.

By deﬁnition, consumption goods or services provide utility.
Utility is simply the generic, technical term preferred by economists for the satisfaction, beneﬁt or welfare that people derive from consumption goods or services.

3.4 A CPI is generally understood to be a price index that measures changes in the prices of consumption goods and services acquired, or used, by households.
As explained in Chapter 14, more broadly based price indices can be deﬁned whose scope extends well beyond consumption goods and services, but a CPI is deliber- ately focused on household consumption.

It is, however, possible to deﬁne a CPI that includes the prices of phy- sical assets such as land or dwellings purchased by 3 households.
In the case of owner-occupied dwellings, a key issue is whether to include in the CPI the imputed rents for the ﬂows of housing services provided by the dwellings, or alternatively whether to include the prices of the dwellings themselves in the index (notwithstanding the fact that they are treated as ﬁxed assets and not consumption goods in the system of national accounts (SNA)).

Views diﬀer on this issue.


In any case, purchases of ﬁnancial assets, such as bonds or shares, are excluded because ﬁnancial assets are not goods or services of any kind and are not used to satisfy the personal needs or wants of household members.

Financial transactions do not change wealth as one type of ﬁnancial asset is simply exchanged for another type of ﬁnancial asset.
For exam- ple, when securities are purchased, money is exchanged for a bond or share; or alternatively, when a debt is incurred, money is received in exchange for the creation of a liability.

3.5 Although, by deﬁnition, a CPI is conﬁned to the prices of goods and services consumed by households, it does not necessarily follow that CPIs have to cover all households or all the goods and services they consume.
For example, it might be decided to exclude publicly provided goods which households do not pay for.

Many decisions have to be taken about the precise scope of a CPI even though the general purpose of the index may be determined.

These issues are explored in this and the following chapter.

Alternative consumption aggregates 3.6 As already noted, the concept of consumption is not a precise one and may be interpreted in diﬀerent ways.
In this section, a hierarchy of diﬀerent consump- tion concepts and aggregates is examined.

3.7 Households may acquire goods and services for purposes of consumption in four main ways: – they may purchase them in monetary transactions; – they may produce them themselves for their own consumption; – they may receive them as payments in kind through barter transactions, particularly as remuneration in kind for work done; – they may receive them as free gifts, or transfers, from other economic units.
3.8 The broadest concept of consumption for CPI purposes would be a price index embracing all four categories of consumption goods and services listed above.

This set of consumption goods and services may 39 CONSUMER PRICE INDEX MANUAL: THEORY AND PRACTICE be described as total acquisitions.

Total acquisitions are equivalent to the total actual individual consumption of households as deﬁned in the SNA (see Chapter 14).
It should be noted that total acquisitions constitute a broader concept of consumption than total consump- tion expenditures.

Acquisitions and expenditures 3.9 Expenditures are made by the economic units who pay for the goods and services: in other words, who bear the costs.

However, many of the goods and services consumed by households are ﬁnanced and paid for by government units or non-proﬁt institutions.

They are mostly services such as education, health, housing and transport.
Individual goods and services provided free of charge, or at nominal prices, to individual households by governments or non-proﬁt institutions are described as social transfers in kind.

They may make a substantial contribution to the welfare or standard of living of the individual households that receive them.

(Social trans- fers in kind do not include collective services provided by governments to the community as whole, such as public administration and defence.)
3.10 The expenditures on social transfers in kind are incurred by the governments or non-proﬁt institu- tions that pay for them and not by the households that consume them.

It could be decided that the CPI should be conﬁned to consumption expenditures incurred by households, in which case free social transfers in kind would be excluded from the scope of the index.
Even if they were to be included, they can be ignored in practice when they are provided free, on the grounds that house- holds incur zero expenditures on them.

Of course, their prices are not zero from the perspective of the units that ﬁnance the social transfers, but the relevant prices for a CPI are those payable by the households.
3.11 Social transfers cannot be ignored, however, when governments and non-proﬁt institutions decide to introduce charges for them, a practice that has become increasingly common in many countries.

For example, if the CPI is intended to measure the change in the total value of a basket of consumption goods and services that includes social transfers, increases in their prices from zero to some positive amount increase the cost of the basket and ought to be captured by a CPI.
Monetary versus non-monetary expenditures 3.12 A distinction may also be drawn between monetary and non-monetary expenditures depending on the nature of the resources used to pay for the goods and services.

A monetary expenditure occurs when a house- hold pays in cash, by cheque or credit card, or otherwise incurs a ﬁnancial liability to pay, in exchange for the acquisition of a good or service.
Non-monetary expen- ditures occur when households do not incur a ﬁnancial liability but bear the costs of acquiring the goods or services in some other way.

3.13 Non-monetary expenditures.

Payments may be made in kind rather than cash, as in barter transactions.

The goods and services oﬀered as payment in barter 40 transactions are equivalent to negative expenditures and their price changes should, in principle, carry negative weights in a CPI.
If the price of goods sold increases, the household is better oﬀ.

However, as the two sides of a barter transaction should in principle be equal in value, the net expenditure incurred by two households engaged in barter should be zero.

Barter transactions between households may therefore be ignored in practice for CPI purposes.

3.14 Households also incur non-monetary expendi- tures when household members receive goods and ser- vices from their employers as remuneration in kind.
The employees pay for the goods and services with their own labour rather than cash.

Consumption goods and ser- vices received as remuneration in kind can, in principle, be included in a CPI using the estimated prices that would be payable for them on the market.

3.15 A third important category of non-monetary expenditure occurs when households consume goods and services that they have produced themselves.
The households incur the costs, while the expenditures are deemed to occur when the goods and services are con- sumed.

Own account expenditures of this kind include expenditures on housing services produced for their own consumption by owner-occupiers.

The treatment of goods and services produced for own consumption raises important conceptual issues that are discussed in more detail below.

3.16 Monetary expenditures.


The narrowest concept of consumption that could be used for CPI purposes is one based on monetary expenditures only.

Such an aggregate would exclude many of the goods and services actually acquired and used by households for purposes of consumption.

Only monetary expenditures generate the monetary prices needed for CPI purposes.
The prices of the goods and services acquired through non-monetary expenditures can only be imputed on the basis of the prices observed in monetary transactions.

Imputed prices do not generate more price information.

Instead, they aﬀect the weighting attached to monetary prices by increasing the weight of those monetary prices which are used to value non-monetary expenditures.
3.17 If the main reason for compiling a CPI is the measurement of inﬂation, it may be decided to restrict the scope of the index to monetary expenditures only, especially since non-monetary expenditures do not gen- erate any demand for money.

Harmonized Indices of Consumer Prices (HICPs), used to measure inﬂation within the European Union, are conﬁned to monetary expenditures (see Annex 1).
Acquisitions and uses 3.18 It has been customary in the literature on CPIs to draw a distinction between acquisitions of consump- tion goods and services by households and their sub- sequent use to satisfy their households’ needs or wants.

Consumption goods are typically acquired at one point of time and used at some other point of time, often much later, or they may be used repeatedly, or even con- tinuously, over an extended period of time.
The times of acquisition and use nevertheless coincide for many services, although there are other kinds of services that provide lasting beneﬁts and are not used up at the time they are provided.

3.19 The time at which a good is acquired is the moment at which ownership of the good is transferred to the consumer.

In a market situation, it is the moment at which the consumer incurs a liability to pay, either in cash or in kind.
The time at which a service is acquired is not so easy to determine precisely as the provision of a service does not involve any exchange of ownership.

Instead, it typically leads to some improvement in the condition of the consumer.

A service is acquired by the consumer at the same time that the producer provides it and the consumer accepts a liability to pay.

3.20 In a market situation, therefore, the time of acquisition for both goods and services is the time at which the liability to pay is incurred.
When payments are not made immediately in cash, there may be a signiﬁcant lapse of time before the consumer’s bank account is debited for a purchase settled by cheque, by credit card or similar arrangement.

The times at which these debits are eventually made depend on administrative con- venience and on the particular ﬁnancial and institutional arrangements in place.

They have no relevance to the time of recording the transactions or the prices.
3.21 The distinction between time of acquisition and time of use is particularly important for durable goods and certain kinds of services.

Durables and non-durables 3.22 Goods.

A ‘‘non-durable’’ good would be better described as a single use good.

For example, food and drink are used once only to satisfy hunger or thirst.

Heating oil, coal or ﬁrewood can be burnt once only, but they are nevertheless extremely durable physically and can be stored indeﬁnitely.
Households may hold sub- stantial stocks of so-called non-durables, such as many foodstuﬀs and fuel, especially in periods of political or economic uncertainty.

3.23 Conversely, the distinguishing feature of con- sumer durables, such as furniture, household equipment or vehicles, is that they are durable under use.

They can be used repeatedly or continuously to satisfy consumers’ needs over a long period of time, possibly many years.
For this reason, a durable is often described as provid- ing a ﬂow of ‘‘services’’’ to the consumer over the period it is used (see also Box 14.3 of Chapter 14).

There is a close parallel between the deﬁnitions of consumer dur- ables and ﬁxed assets.

Fixed assets are goods that are used repeatedly or continuously over long periods of time in processes of production: for example, buildings or other structures, machinery and equipment.
A list of the diﬀerent kinds of consumer durables distinguished in the Classiﬁcation of Individual Consumption according to Purpose (COICOP) is given below.

Some durables last much longer than others, the less durable ones being described as ‘‘semi-durables’’ in COICOP, for example clothing.

Dwellings are not classiﬁed as consumer dur- ables in COICOP.

They are treated as ﬁxed assets and not consumption goods and therefore fall outside the scope of COICOP.
However, the housing services CONCEPTS AND SCOPE produced and consumed by owner-occupiers are inclu- ded in COICOP and classiﬁed in the same way as the housing services consumed by tenants.

3.24 Services.

Consumers may continue to beneﬁt, and derive utility, from some services long after they were provided because they bring about substantial, long-lasting or even permanent improvements in the condition of the consumers.
The quality of life of persons receiving medical treatments such as hip replacements or cataract surgery, for example, is substantially and per- manently improved.

Similarly, consumers of educational services can beneﬁt from them over their entire lifetimes.

it may be some analytical purposes, appropriate to treat certain kinds of services, such as education and health, as the service equivalents of dur- able goods.
Expenditures on such services can be viewed as investments that augment the stock of human capital.

Another characteristic that education and health ser- vices share with durable goods is that they are often so expensive that their purchase has to be ﬁnanced by borrowing or by running down other assets.
3.25 For Consumer price indices based on acquisitions and uses 3.26 The distinction between the acquisition and the use of a consumption good or service has led to two diﬀerent concepts of a CPI being proposed.

 A CPI may be intended to measure the average change between two time periods in the prices of the con- sumption goods and services acquired by households.
Alternatively, a CPI may be intended to measure the average change between two time periods in the prices of the consumption goods and services used by house- holds to satisfy their needs and wants.

3.27 Flows of acquisitions and uses may be very like diﬀerent for durables.

Acquisitions of durables, producer capital goods, are liable to ﬂuctuate, depending on the general state of the economy, whereas the using up of the stock of durables owned by households tends be a gradual and smooth process.
A CPI based on the uses approach requires that the index should measure period- to-period changes in the prices of the ﬂows of services provided by the durables.

As explained in Chapter 23, the value of the ﬂow of services from a durable may be estimated by its ‘‘user cost’’, which consists essentially of the depreciation on the asset (at current prices) plus the interest cost.
The inclusion of the interest cost as well as the depreciation means that, over the long term, the weight given to durables is greater than when they are measured simply by acquisitions.

In principle, the ﬂows of services, or beneﬁts, derived from major educational and medical expenditures might also be estimated on the basis of user costs.
3.28 When durables are rented on the market, the rentals have to cover not only the values of the service ﬂows but additional costs such as administration and management, repairs and maintenance, and overheads.
For example, the amount payable to use a washing machine in a launderette has to cover the costs of the room space in which the machine is housed, electricity, 41 CONSUMER PRICE INDEX MANUAL: THEORY AND PRACTICE repairs and maintenance, the wages of supervisory staﬀ, and so on, as well as the services provided by the machine itself.

Similarly, the rentals payable for car hire may signiﬁcantly exceed the cost of the service ﬂow provided by the car on its own.
In both cases, the cus- tomer is buying a bundle of services that includes more than just the use of the durable good.

3.29 Estimating the values and the prices of the ﬂows of services provided by the stock of durables owned by households is diﬃcult, whereas expenditures on durables are easily recorded, as are also the prices at which they are purchased.

Partly because of these practical mea- surement diﬃculties, CPIs have, up to now, been based largely or entirely on the acquisitions approach.
Simi- larly, national accounts tend to record expenditures on, or acquisitions of, durables rather than the ﬂows of services they provide.

As already noted, dwellings are treated as ﬁxed assets and not consumer durables in the SNA.

The treatment of owner-occupied housing is con- sidered separately below.

Basket indices and cost of living indices 3.30 A fundamental conceptual distinction may be drawn between a basket index and a cost of living index.
In a CPI context, a basket index is an index that mea- sures the change between two time periods in the total expenditure needed to purchase a given set, or basket, of consumption goods and services.

It is called a ‘‘Lowe index’’ in this manual.

A cost of living index (COLI) is an index that measures the change in the minimum cost of maintaining a given standard of living.
Both indices therefore have very similar objectives in that they aim to measure the change in the total expenditure needed to purchase either the same basket or two baskets whose composition may diﬀer somewhat but between which the consumer is indiﬀerent.

Lowe indices 3.31 CPIs are almost invariably calculated as Lowe indices in practice.

Their properties and behaviour are described in detail in various chapters of this manual.
The operational target for most CPIs is to measure the change over time in the total value of some speciﬁed basket of consumption goods and services purchased, or acquired, by some speciﬁed group of households in some speciﬁed period of time.

The meaning of such an index is clear.

It is, of course, necessary to ensure that the selected basket is relevant to the needs of users and also kept up to date.
The basket may be changed at regular intervals and does not have to remain ﬁxed over long periods of time.

The determination of the basket is considered in more detail later in this chapter and in the following one.

Cost of living indices 3.32 The economic approach to index number the- ory treats the quantities consumed as being dependent on the prices.

Households are treated as price takers who are assumed to react to changes in relative prices by adjusting the relative quantities they consume.
A 42 basket index that works with a ﬁxed set of quantities fails to allow for the fact that there is a systematic ten- dency for consumers to substitute items that have become relatively cheaper for those that have become relatively dearer.

A cost of living index based on the economic approach does take this substitution eﬀect into account.
It measures the change in the minimum expenditure needed to maintain a given standard when utility-maximizing consumers adjust their patterns of purchases in response to changes in relative prices.

In contrast to a basket index, the baskets in the two periods in a cost of living index will generally not be quite the same in the two periods because of these substitutions.

3.33 The properties and behaviour of cost of living in indices, or COLIs, are explained in some detail Chapter 17.
A summary explanation has already been given in Chapter 1.

The maximum scope of a COLI would be the entire set of consumption goods and services consumed by the designated households from which they derive utility.

It includes the goods and services received free as social transfers in kind from govern- ments or non-proﬁt institutions.
Because COLIs mea- sure the change in the cost of maintaining a given standard of living or level of utility, they lend themselves to a uses rather than an acquisitions approach, as utility is derived not by acquiring a consumer good or service but by using it to satisfy personal needs and wants.
3.34 Welfare may be interpreted to mean not only economic welfare, that is the utility that is linked to economic activities such as production, consumption and working, but also general well-being associated with other factors such as security from attack by others.

It may not be possible to draw a clear distinction between economic and non-economic factors, but it is clear that total welfare is only partly dependent on the amount of goods and services consumed.
3.35 Conditional and unconditional cost of living indices.

In principle, the scope of a COLI is inﬂuenced by whether or not it is intended to be a conditional and unconditional cost of living index.
The total welfare of a household depends on a string of non-economic factors such as the climate, the state of the physical, social and political environment, the risk of being attacked either by criminals or from abroad, the incidence of diseases, and so on, as well as by the quantities of goods and services consumed.
An unconditional cost of living index measures the change in the cost to a household of maintaining a given level of total welfare allowing the non-economic factors to vary as well as the prices of consumption goods and services.

If changes in the non- economic factors lower welfare, then some compensat- ing increase in the level of consumption will be needed in order to maintain the same level of total welfare.
An adverse change in the weather, for example, requires more fuel to be consumed to maintain the same level of comfort as before.

The cost of the increased quantities of fuel consumed drives up the unconditional cost of living index, irrespective of what has happened to prices.

There are countless other events that can impact on an un- conditional cost of living index, from natural disasters such as earthquakes to man-made disasters such as Chernobyl or acts of terrorism.
3.36 While there may be interest in an uncondi- tional cost of living index for certain analytical and policy purposes, it is deﬁned in such a way that it is deliberately intended to measure the eﬀects of many other factors besides prices.

If the objective is to mea- sure the eﬀects of price changes only, the non-price factors must be held constant.
Given that a cost of living index is meant to serve as a consumer price index, its scope must be restricted to exclude the eﬀects of events other than price changes.

A conditional cost of living index is deﬁned as the ratio of the minimum expenditures needed to maintain a given level of utility, or welfare, in response to price changes, assuming that all the other factors aﬀecting welfare remain constant.
It is conditional not only on a particular standard of living and set of preferences, but also on a particular state of the non-price factors aﬀecting welfare.

COLIs in this manual are to be understood as conditional cost of living indices.

index.

An unconditional 3.37 A conditional COLI should not be viewed as second best.
An unconditional COLI is a more com- prehensive cost of living index than a conditional COLI, but it is not a more comprehensive price index than a index does not conditional include more price information than a conditional index and it does not give more insight into the impact of price changes on households’ welfare.

On the contrary, the impact of the price changes is diluted and obscured as more variables impacting on welfare are included within the scope of the index.
3.38 Lowe indices, including Laspeyres and Paasche, are also conditional, being dependent on the choice of basket.

The fact that the value of a basket index varies in predictable ways according to the choice of basket has generated much of the large literature on index number theory.

Conceptually, Lowe indices and condi- tional COLIs have much in common.
A Lowe index measures the change in the cost of a speciﬁed basket of goods and services, whereas a conditional COLI mea- sures the change in the cost of maintaining the level of utility associated with some speciﬁed basket of goods and services, other things being equal.
Expenditures and other payments outside the scope of consumer price indices 3.39 Given that, conceptually, most CPIs are designed to measure changes in the prices of consump- tion goods and services, it follows that purchases of items that are not goods and services fall outside the intended scope of a CPI: for example, purchases of bonds, shares or other ﬁnancial assets.
Similarly, payments that are not even purchases because nothing is received in exchange fall outside the index: for example, payments of income taxes or social security contributions.

3.40 The implementation of these principles is not always straightforward, as the distinction between an expenditure on a good or service and other payments may not always be clear cut in practice.
A number of conceptually diﬃcult cases, including some borderline CONCEPTS AND SCOPE cases of a possibly controversial nature, are examined below.

Transfers 3.41 The deﬁnition of a transfer is a transaction in which one unit provides a good, service or asset to another without receiving any good, service or asset in return: i.e., transactions in which there is no counter- part.

Transfers are unrequited.


As no good or service of any kind is acquired by the household when it makes a transfer, the transfer must be outside the scope of a CPI.

The problem is to determine whether or not certain kinds of transactions are in fact transfers, a problem common to both CPIs and national accounts.

3.42 Social security contributions and taxes on income and wealth.
As households do not receive any speciﬁc, individual good or service in return for the payment of social security contributions, they are treated as transfers that are outside the scope of CPIs.

Similarly, all pay- ments of taxes based on income or wealth (the ownership of assets) are outside the scope of a CPI since they are unrequited compulsory transfers to government.

Property taxes on dwellings (commonly levied as local authority taxes or rates) are outside the scope.
It may be noted, however, that unrequited compulsory transfers could be incorporated within an unconditional COLI or within a more broadly deﬁned conditional COLI that allows for changes in some other factors besides changes in the prices of consumption goods and services.

3.43 Licences.
3.43 Licences.

Households have to pay to obtain various kinds of licences and it is often not clear whether they are simply taxes under another name or whether the government agency providing the licence provides some kind of service in exchange, for example by exer- cising some supervisory, regulatory or control function.

In the latter case, they could be regarded as purchases of services.
Some cases are so borderline that they have been debated for years by taxation experts under the aegis of the International Monetary Fund (IMF) and other international agencies without reaching consensus.

The experts therefore agreed to settle on a number of conventions based on practices followed in the majority of countries.

It is appropriate to make use of these conventions for CPI, and also national accounts, pur- poses.
These conventions are listed in the IMF’s Govern- ment Finance Statistics (IMF, 2001) and have also been adopted in SNA 1993.

3.44 Payments by households for licences to own or use certain goods or facilities are, by convention, classi- ﬁed as consumption expenditures, not transfers, and are thus included within the scope of a CPI.

For example, licence fees for radios, televisions, driving, ﬁrearms, and so on, as well as fees for passports, are included.
On the other hand, licences for owning or using vehicles, boats and aircraft, and for hunting, shooting and ﬁshing are conventionally classiﬁed as direct taxes and are therefore outside the scope of CPIs.

Many countries, however, do include taxes for private vehicle use as they regard them as taxes on consumption for CPI purposes.
As the actual circumstances under which licences are issued, and the conditions attaching to them, can vary signiﬁcantly from 43 CONSUMER PRICE INDEX MANUAL: THEORY AND PRACTICE country to country, statistical oﬃces may wish to deviate from the proposed conventions in some instances.

In general, however, it seems appropriate to make use of conventions internationally agreed by the relevant experts.

3.45 Gifts and subscriptions.

Gifts are transfers, by deﬁnition, and thus outside the scope of a CPI.
Pay- ments of subscriptions or donations to charitable orga- nizations for which no easily identiﬁable services are received in return are also transfers.

On the other hand, payments of subscriptions to clubs and societies, includ- ing charities, which provide their members with some kind of service (e.g., regular meetings, magazines, etc.)

can be regarded as ﬁnal consumption expenditures and included in a CPI.

3.46 Tips and gratuities.


Non-compulsory tips or gratuities are gifts that are outside the scope of a CPI.

There may be cases, however, where, although tips are not compulsory, it can be very diﬃcult to obtain a good or service without some form of additional payment, in which case this payment should be included in the expenditure on, and the price of, the good or service in question.

Insurance 3.47 There are two main types of insurance, life and non-life.
In both cases the premiums have two compo- nents.

One is a payment for the insurance itself, often described as the net premium, while the other is an implicit service charge payable to the insurance enter- prise for arranging the insurance: i.e., a fee charged for calculating the risks, determining the premiums, admin- istering the collection and investment of premiums, and the payment of claims.

3.48 The implicit service charge is not directly observable.
It is an integral part of the gross premium that is not separately identiﬁed in practice.

As a pay- ment for a service it falls within the scope of a CPI, but it is diﬃcult to estimate.

3.49 In the case of non-life insurance, the net pre- mium is essentially a transfer that goes into a pool cov- ering the collective risks of policy holders as a whole.

As a transfer, it falls outside the scope of a CPI.

In the case of life insurance, the net premium is essentially a form of ﬁnancial investment.
It constitutes the purchase of a ﬁnancial asset, which is also outside the scope of a CPI.

3.50 Finally, it may be noted that when insurance is arranged through a broker or agent separate from the insurance enterprise, the fees charged by the brokers or agents for their services are included within the scope of the CPI, over and above the implicit service charges made by the insurers.
Gambling 3.51 The amounts paid for lottery tickets or placed in bets also consist of two elements that are usually not separately identiﬁed – the payment of an implicit service charge (part of consumption expenditures) and a current transfer that enters the pool out of which the winnings are paid.

Only the implicit or explicit service charges payable to the organizers of the gambling fall within the 44 scope of a CPI.
The service charges are usually calculated at an aggregate level as the diﬀerence between payables (stakes) and receivables (winnings).

Transactions in ﬁnancial assets 3.52 Financial assets are not consumption goods or services.

The creation of ﬁnancial assets/liabilities, or their extinction, e.g., by lending, borrowing and repay- ments, are ﬁnancial transactions that are quite diﬀerent from expenditures on goods and services and take place independently of them.
The purchase of a ﬁnancial asset is obviously not expenditure on consumption, being a form of ﬁnancial investment.

3.53 Some ﬁnancial assets, notably securities in the form of bills, bond and shares, are tradable and have market prices.

They have their own separate price indi- ces, such as stock market price indices.

3.54 Many of the ﬁnancial assets owned by house- holds are acquired indirectly through the medium of pension schemes and life insurance.
Excluding the service charges, pension contributions by households are similar to payments for life insurance premiums.

They are essentially forms of investment made out of saving, and are thus excluded from CPIs.

In contrast, the explicit or implicit fees paid by households for the services rendered by ﬁnancial auxiliaries such as brokers, banks, insurers (life and non-life), pension fund managers, ﬁnancial advisers, accountants, and so on, are within the scope of a CPI.
Payments of such fees are simply purchases of services.

Purchases and sales of foreign currency 3.55 Foreign currency is a ﬁnancial asset.

Purchases and sales of foreign currency are therefore outside the scope of CPIs.

Changes in the prices payable, or receiv- able, for foreign currencies resulting from changes in exchange rates are not included in CPIs.
In contrast, the service charges made by foreign-exchange dealers are included within the scope of CPIs when households acquire foreign currency for personal use.

These charges include not only explicit commission charges but also the margins between the buying or selling rates oﬀered by the dealers and the average of the two rates.
Payments, ﬁnancing and credit 3.56 Conceptually, the time at which an expenditure is incurred is the time at which the purchaser incurs a liability to pay: that is, when the ownership of the good changes hands or the service is provided.

The time of payment is the time at which the liability is extinguished.
The two may be simultaneous when payment is made immediately in cash, i.e., notes or coin, but the use of cheques, credit cards and other forms of credit facilities means that it is increasingly common for the payment to take place some time after the expenditure occurs.

A further complication is that payments may be made in stages, with a deposit payable in advance.
Given the time lags and complexity of ﬁnancial instruments and institutional arrangements, it may be diﬃcult to deter- mine exactly when payment takes place.

The time may even be diﬀerent from the standpoint of the purchaser and the seller.

3.57 For consistency with the expenditure data used as weights in CPIs, the prices should be recorded at the times at which the expenditures actually take place.

This is consistent with an acquisitions approach.


Financial transactions and borrowing 3.58 Some individual expenditures may be very large: for example, the purchase of expensive medical treatment, a large durable good, or an expensive holi- day.

If the household does not have suﬃcient cash, or does not wish to pay the full amount immediately in cash, various options are open.

 The purchaser may borrow from a bank, moneylender or other ﬁnancial institution.
The purchaser may use a credit card.

 The seller may extend credit to the purchaser, or the seller may arrange for a third party, some kind of ﬁnancial institution, to extend credit to the purchaser.

The creation of a ﬁnancial asset/liability 3.59 When a consumer borrows to purchase a good or service, two distinct transactions are involved: the purchase of the good or service, and the borrowing of the requisite funds.
The latter is a purely ﬁnancial trans- action between a creditor and a debtor in which a new ﬁnancial asset/liability is created.

This ﬁnancial trans- action is outside the scope of a CPI.

As already noted, a ﬁnancial transaction does not change wealth and there is no consumption involved.

A ﬁnancial transaction merely rearranges the individual’s asset portfolio by exchanging one type of asset for another.

For example, when a loan is made, the lender exchanges cash for a ﬁnancial claim over the debtor.
Similarly, the borrower acquires cash counterbalanced by the creation of an equal ﬁnan- cial liability.

Such transactions are irrelevant for CPI purposes.

3.60 In general, when a household borrows from ﬁnancial institutions, including moneylenders, the bor- rowed funds may be used for a variety of purposes including the purchase of assets such as dwellings or ﬁnancial assets (for example, bonds or shares), as well as the purchase of expensive goods and services.
Similarly, the credit extended to the holder of a credit card can be used for a variety of purposes.

In itself, the creation of a ﬁnancial asset and liability by new borrowing has no impact on a CPI.

There is no good or service acquired, no expenditure and no price.

3.61 It should be noted that interest payments are not themselves ﬁnancial transactions.

The payment of interest is quite diﬀerent from the borrowing, lending or other ﬁnancial transactions that give rise to it.
Interest is considered separately below.

3.62 Hire purchase and mortgage loans must be treated consistently with other loans.

The fact that cer- tain loans are conditional on the borrower using the funds for a particular purpose does not aﬀect the CONCEPTS AND SCOPE treatment of the loan itself.

Moreover, conditional loans are by no means conﬁned to the purchase of durable goods on ‘‘hire purchase’’.
Conditional personal loans may be made for other purposes, such as large expen- ditures on education or health.

In each case, the con- tracting of the loan is a separate transaction from the expenditure on the good or service and must be dis- tinguished from the latter.

The two transactions may involve diﬀerent parties and may take place at quite diﬀerent times.
3.63 Although the provision of ﬁnance is a separate transaction from the purchase of a good or service for which it is used, it may aﬀect the price paid.

Each case needs to be carefully considered.

For example, suppose the seller agrees to defer payment for one year.

The seller appears to make an interest free loan for a year, but this is not the economic reality.

The seller makes a loan but it is not interest free.

Nor is the amount lent equal to the ‘‘full’’ price.
Implicitly, the purchaser issues a short-term bill to the seller to be redeemed one year later and uses the cash received from the seller to pay for the good.

However, the present value of a bill at the time it is issued is its redemption value discounted by one year’s interest.

The amount payable by the purchaser at the time the purchase of the good actually takes place is the present discounted price of the bill and not the full redemption price to be paid one year later.
It is this discounted price that should be recorded for CPI pur- poses.

The diﬀerence between the discounted price and the redemption price is, of course, the interest that the purchaser implicitly pays on the bill over the course of the year.

This way of recording corresponds to the way in which bills and bonds are actually valued on ﬁnancial markets and also to the way in which they are recorded in both business and economic accounts.
Deferring pay- ments in the manner just described is equivalent to a price reduction and should be recognized as such in CPIs.

The implicit interest payment is not part of the price.

Instead, it reduces the price.

This example shows that in certain circumstances the market rate of interest can aﬀect the price payable, but it depends on the exact circumstances of the credit arrangement agreed between the seller and the purchaser.

Each individual case needs to be carefully considered on its merits.
3.64 This case needs to be clearly distinguished from hire purchase, considered in the next section, when the purchaser actually pays the full price and borrows an amount equal to the full price while contracting to make explicit interest payments in addition to repaying the amount borrowed.

Hire purchase In the case of a durable good bought on hire 3.65 purchase, it is necessary to distinguish the de facto, or economic, ownership of the good from the legal ownership.
The time of acquisition is the time the hire purchase contract is signed and the purchaser takes possession of the durable.

From then onwards, it is the purchaser who uses it and derives the beneﬁt from its use.

The purchasing household becomes the de facto owner at the time the good is acquired, even though legal 45 CONSUMER PRICE INDEX MANUAL: THEORY AND PRACTICE ownership may not pass to the household until the loan is fully repaid.
therefore, 3.66 By convention, the purchasing household is treated as buying the good at the time possession is taken and paying the full amount in cash at that point.

At the same time, the purchaser borrows, either from the seller or some ﬁnancial institution speci- ﬁed by the seller, a sum suﬃcient to cover the purchase price and the subsequent interest payments.

The diﬀer- ence between the cash price and the sum total of all the payments to be made is equal to the total interest pay- able.
The relevant price for CPI purposes is the cash price payable at the time the purchase takes place, whether or not the purchase is facilitated by some form of borrow- ing.

The treatment of hire purchase is the same as that of ‘‘ﬁnancial leasing’’ whereby ﬁxed assets, such as aircraft, used for purposes of production are purchased by a ﬁnancial institution and leased to the producer for most or all of the service life of the asset.
This is essentially a method of ﬁnancing the acquisition of an asset by means of a loan and needs to be distinguished from operational leasing such as hiring out cars for short periods of time.

The treatment of hire purchase and ﬁnancial leasing outlined here is followed in both business and economic accounting.

Interest payments 3.67 The treatment of interest payments on the various kinds of debt that households may have incur- red raises both conceptual and practical diﬃculties.
Nominal interest is a composite payment covering four main elements whose mix may vary considerably: The ﬁrst component is the pure interest charge: i.e., the interest that would be charged if there were perfect capital markets and perfect information.

 The second component is a risk premium that depends on the creditworthiness of the individual borrower.

It can be regarded as a built-in insurance charge under uncertainty against the risk of the debtor defaulting.
The third component is a service charge incurred when households borrow from ﬁnancial institutions that make a business of lending money.

 Finally, when there is inﬂation, the real value of a loan ﬁxed in monetary terms (that is, its purchasing power over real goods and services) declines with the rate of inﬂation.

However, creditors are able to oﬀset the real holding, or capital, losses they expect to incur by charging appropriately high rates of nominal interest.
For this reason, nominal interest rates vary directly with the rate of general inﬂation, a universally fam- iliar phenomenon under inﬂationary conditions.

In these circumstances, the main component of nominal interest may therefore be the built-in payment of compensation from the debtor to the creditor to oﬀset the latter’s real holding loss.

When there is very high inﬂation it may account for almost all of the nominal interest charged.
3.68 The treatment of the ﬁrst component, pure interest, is somewhat controversial but this component may account for only a small part of the nominal 46 interest charged.

The treatment of the second compo- is also insurance against the risk of default, nent, somewhat controversial.

3.69 The fourth component, the payment of com- pensation for the creditor’s real holding loss, is clearly outside the scope of a CPI.

It is essentially a capital transaction.


It may account for most of nominal interest under inﬂationary conditions.

3.70 The third component constitutes the purchase of a service from ﬁnancial institutions whose business it is to make funds available to borrowers.

It is known as the implicit service charge and clearly falls within the scope of a CPI.

It is included in COICOP.


The service charge is not conﬁned to loans made by ‘‘ﬁnancial intermediaries’’, institutions that borrow funds in order to lend them to others.

Financial institutions that lend out of their own resources provide the same kind of services to borrowers as ﬁnancial intermediaries.

When sellers lend out of their own funds, they are treated as implicitly setting up their own ﬁnancial institution that operates separately from their principal activity.
The rates of interest of ﬁnancial institutions also include implicit service charges.

Because some capital markets tend to be very imperfect and most households may not have access to proper capital markets, many lenders are eﬀectively monopolists who charge very high prices for the services they provide, for example village money- lenders in many countries.

3.71 It is clear that interest payments should not be treated as if they were just pure interest or even pure interest plus a risk premium.
It is very diﬃcult to dis- entangle the various components of interest.

It may be practically impossible to make realistic and reliable estimates of the implicit service charges embodied in most interest payments.

Moreover, for CPI purposes it is necessary to estimate not only the values of the service charges but changes in the prices of the services over time.
Given the complexity of interest ﬂows and the fact that the diﬀerent ﬂows need to be treated diﬀerently, there seems to be little justiﬁcation for including pay- ments of nominal interest in a CPI, especially in inﬂa- tionary conditions.

Household production 3.72 Households can engage in various kinds of productive activities that may be either aimed at the market or intended to produce goods or services for own consumption.
Business activities 3.73 Households may engage in business or commer- cial activities such as farming, retail trading, construc- tion, the provision of professional or ﬁnancial services, and so on.

Goods and services that are used up in the process of producing other goods and services for sale on the market constitute intermediate consumption.

They are not part of the ﬁnal consumption of households.

The prices of intermediate goods and services purchased by households are not to be included in CPIs.
In practice, it is often diﬃcult to draw a clear distinction between intermediate and ﬁnal consumption, as the same goods may be used for either purpose.

Consumption of own produce 3.74 Households do not in fact consume directly all of the goods and services they acquire for purposes of consumption.

Instead, they use them as inputs into the production of other goods or services which are then used to satisfy their needs and wants.

There are numer- ous examples.


For example, basic foodstuﬀs such as ﬂour, cooking oils, raw meat and vegetables may be processed into bread, cakes or meals with the assistance of other inputs including fuels, the services provided by consumer durables, such as fridges and cookers, and the labour services of members of the household.

Inputs of materials, equipment and labour are used to clean, maintain and repair dwellings.
Inputs of seeds, fertilizers, insecticides, equipment and labour are used to produce vegetables or ﬂowers, and so on.

3.75 Some of the production activities taking place within households’ activities, for example gardening or cooking, may perhaps provide satisfaction in themselves.

Others, such as cleaning, may be regarded as chores that reduce utility.

In any case, the goods or services used as inputs into these productive activities do not provide utility in themselves.
Again, there are numerous exam- ples of such inputs: raw foodstuﬀs that are unsuitable for eating without being cooked; cleaning materials; fuels such as coal, gas, electricity or petrol; fertilizers; the services of refrigerators and freezers; and so on.

3.76 Utility is derived from consuming the outputs from household production undertaken for own con- sumption.

It is necessary, therefore, to decide whether a CPI should try to measure the changes in the prices of the outputs, rather than the inputs.
In principle, it seems desirable to measure the output prices, but there are serious objections to this procedure.

3.77 On a conceptual level, it is diﬃcult to decide what are the real ﬁnal outputs from many of the more nebulous household production activities.

It is particu- larly diﬃcult to specify exactly what are the outputs from important service activities carried out within house- holds, such as child care or care of the sick or elderly.
Even if they could be satisfactorily identiﬁed, concep- tually they would have to be measured and priced.

There are no prices to be observed, as there are no sales transactions.

Prices would have to be imputed for them and such prices would be not only hypothetical but inevitably very speculative.

Their use in CPIs is not a realistic possibility in general and almost certainly would not be acceptable to most users who are primarily interested in the market prices paid by households.
3.78 The practical alternative is to treat the goods and services acquired by households on the market for use as inputs into the various kinds of household pro- duction activities as if they were themselves ﬁnal con- sumer goods and services.

They provide utility indirectly, assuming that they are used exclusively to produce goods and services that are directly consumed by households.
This is the practical solution that is generally adopted not only in CPIs but also in national accounts, where CONCEPTS AND SCOPE 3.79 Subsistence household expenditures on such items are classiﬁed as ﬁnal consumption.

Although this seems a simple and conceptually acceptable solution to an otherwise intract- able problem, exceptions may be made for one or two kinds of household production that are particularly important and whose outputs can readily be identiﬁed.

agriculture.
agriculture.

national accounts, an attempt is made to record the value of the agricultural output produced for own consumption.

In some countries, subsistence agriculture may account for a large part of the production and consumption of agricultural produce.

The national accounts require such outputs to be valued at their market prices.

It is doubtful whether it is appropriate to try to follow this procedure for CPI purposes.
the In 3.80 A CPI may record either the actual input prices or the imputed output prices, but not both.

If the imputed output prices for subsistence agriculture are included in a CPI, the prices of the purchased inputs should be excluded.

This could remove from the index most of the market transactions made by such house- holds.

Expenditures on inputs may constitute the prin- cipal contact that the households have with the market and through which they experience the eﬀects of inﬂa- tion.
It therefore seems preferable to record the actual prices of the inputs and not the imputed prices of the outputs in CPIs.

3.81 Housing services produced for own consumption.

The treatment of owner-occupied housing is diﬃcult and somewhat controversial.

There may no consensus on what is best practice.

This is discussed in several chapters of this manual, especially in Chapters 10 and 23.
Con- ceptually, the production of housing services for own consumption by owner-occupiers is no diﬀerent from other types of own account production taking place within households.

The distinctive feature of the pro- duction of housing services for own consumption, as compared with other kinds of household production, is that it requires the use of an extremely large ﬁxed asset in the form of the dwelling itself.
In economics, and also national accounting, a dwelling is usually regarded as a ﬁxed asset so that the purchase of a dwelling is classiﬁed as gross ﬁxed capital formation and not as the acquisi- tion of a durable consumer good.

Fixed assets are used for purposes of production, not consumption.

The dwel- ling is not consumed directly.

The dwelling provides a stream of capital services that are consumed as inputs into the production of housing services.
This production requires other inputs, such as repairs, maintenance and insurance.

Households consume the housing services produced as outputs from this production.

3.82 It is important to note that there are two quite distinct service ﬂows involved: One consists of the ﬂow of capital services provided by the dwelling which are consumed as inputs into the production of housing services.
The other consists of the ﬂow of housing services produced as outputs which are consumed by members of the household.

The two ﬂows are not the same.

The value of the output ﬂow will be greater than that of the input ﬂow.

The 47 CONSUMER PRICE INDEX MANUAL: THEORY AND PRACTICE capital services are deﬁned and measured in exactly the same way as the capital services provided by other kinds of ﬁxed assets, such as equipment or structures other than dwellings.
As explained in detail in Chapter 23, the value of the capital services is equal to the user cost and consists primarily of two elements, depreciation and the interest, or capital, costs.

Capital costs are incurred whether or not the dwelling is purchased by borrowing on a mortgage.

When the dwelling is purchased out of own funds, the interest costs represent the opportunity cost of the capital tied up in the dwelling; that is, the foregone interest that could have been earned by investing elsewhere.
3.83 There are two main options for the own- account production and consumption of housing ser- vices in CPIs.

One is to price the output of housing services consumed.

The other is to price the inputs, including the inputs of capital services.

If housing ser- vices are to be treated consistently with other forms of production for own consumption within households, the input approach must be adopted.
The production and consumption of housing services by owner-occupiers may, however, be considered to be so important as to merit special treatment.

3.84 If it is decided to price the outputs, the prices may be estimated using the market rents payable on rented accommodation of the same type.

This is des- cribed as the rental equivalence approach.

One practical problem is that there may be no accommodation of the same type that is rented on the market.
For example, there may be no rental market for rural dwellings in developing countries where most of the housing may actually be constructed by the households themselves.

Another problem is to ensure that the market rents do not include other services, such as heating, that are additional to the housing services proper.
A further problem is that market rents, like the rentals charged when durables are leased, have to cover the operating expenses of the renting agencies as well as the costs of the housing services themselves, and also provide some proﬁt to the owners.

Finally, rented accommodation is inherently diﬀerent from owner-occupied housing in that it may provide the tenants with more ﬂexibility and mobility.

The transaction costs involved in moving house may be much less for tenants.
3.85 In principle, if the output, or rental equiva- lence, approach is adopted then the prices of the inputs into the production of housing services for own con- sumption, such as expenditures on repairs, maintenance and insurance, should not be included as well.

Other- wise, there would be double counting.

3.86 The alternative is to price the inputs into the production of housing services for own consumption in the same way that other forms of production for own consumption within households are treated.
In addition to intermediate expenditures such as repairs, main- tenance and insurance, the costs of the capital services must be estimated and their prices included in the CPI.

The technicalities of estimating the values of the ﬂow of capital services are dealt with in Chapter 23.

As in the case of other types of production for own consumption within households, it is not appropriate to include the 48 estimated costs of the labour services provided by the owners themselves.
3.87 Whether the input or the output approach is adopted, it is diﬃcult to estimate the relevant prices.

The practical diﬃculties experienced may sometimes be so great as to lead compilers and users to query the reli- ability of the results.

There is also some reluctance to use imputed prices in CPIs, whether the prices refer to the inputs or the outputs.

It has therefore been suggested that the attempt to measure the prices of housing service ﬂows should be abandoned.
Instead, it may be preferred to include the prices of the dwellings themselves in the CPI.

In most cases these are observable market prices, although many dwellings, especially in rural areas in developing countries, are also built by their owners, in which case their prices still have to be estimated on the basis of their costs of production.

3.88 Including the prices of dwellings in CPIs involves a signiﬁcant change in the scope of the index.
A dwelling is clearly an asset and its acquisition is capital formation and not consumption.

While the same argu- ment applies to durables, there is a substantial diﬀerence of degree between a household durable and a dwelling, as reﬂected by the considerable diﬀerences in their prices and their service lives.

In principle, therefore, extending the scope of a CPI to include dwellings implies extending the scope of the index to include household gross ﬁxed capital formation.
3.89 The advantage of this solution is that it does not require estimates of either the input or output ser- vice ﬂows, but conceptually it deviates signiﬁcantly from the concept of a CPI as traditionally understood.

In the case of both consumer durables and dwellings, the options are either to record the acquisitions of the assets in the CPIs at their market prices or to record the esti- mated prices of the service ﬂows, but not both.
Just as no service ﬂows from durables are included in CPIs at present because their acquisitions are included, similarly if the prices of dwellings are included in CPIs the service ﬂows would have to be excluded.

As explained in Chapter 23, the acquisitions approach may give insuﬃ- cient weight to durables and dwellings over the long run because it does not take account of the capital costs incurred by the owners of the assets.
Coverage of households and outlets 3.90 The group of households included in the scope of a CPI is often referred to as the ‘‘reference house- holds’’, or the ‘‘reference population’’.

Deﬁnition of household 3.91 For CPI purposes, households may be deﬁned in the same way as in population censuses.
The follow- ing deﬁnition is recommended for use in population censuses (United Nations, 1998a): A household is classiﬁed as either (a) a one person household deﬁned as an arrangement in which one person makes provision for his or her food or other essentials for living without combining with any other person to form part of a multi-person household; or (b) a multi-person household, deﬁned as a group of two or more persons living together who make common provision for food or other essentials for living.
The persons in the group may pool their incomes and have a common budget to a greater or lesser extent; they may be related or unrelated persons or a combination of persons both related and unrelated.

3.92 This deﬁnition is essentially the same as that used in household budget surveys and in the SNA.
The scope of a CPI is usually conﬁned to private households, and excludes institutional households such as groups of persons living together indeﬁnitely in religious institu- tions, residential hospitals, prisons or retirement homes.

Nevertheless, convalescent homes, schools and colleges, the military, and so on are not treated as institutional households; their members are treated as belonging to their private households.
The HICP coverage of house- holds, however, is consistent with the SNA 1993 deﬁ- nition and thus includes institutional households.

Types of household 3.93 In almost all countries, the CPI scope is designed to include as many private households as pos- sible, and is not conﬁned to those belonging to a speciﬁc socio-economic group.

The HICP regulations require that coverage should be of households independent of their income level.
3.94 In some countries, however, extremely wealthy households are excluded for various reasons.

Their expenditures may be considered to be very atypical, while their expenditure data, as collected in household budget surveys, may be unreliable.

The response rates for wealthy households in household budget surveys are usually quite low.

In addition, it may be too costly to collect prices for some of the consumer goods and services purchased exclusively by the wealthy.
Some countries may decide to exclude other kinds of households.

For example, the United Kingdom CPI excludes not only the top 4 per cent of households by income but also households mainly dependent on state pensions, with the net result that roughly 15 per cent of households, and 15 per cent of expenditure, is excluded.

Japan and the Republic of Korea exclude households mainly engaged in agriculture, forestry and ﬁshing, and all one-person households.
Such exclusions aﬀect the expenditure weights to the extent that the patterns of expenditures of the excluded groups diﬀer from those of the rest of the population.

3.95 In addition to a single wide-ranging oﬃcial (headline) CPI relevant to the country as a whole, many countries publish a range of subsidiary indices relating to sub-sectors of the population.
For example, the Czech Republic compiles separate indices for: – all households; – all employees; – employees with children; – low-income employees; – employees, incomplete families; – pensioners; – low-income pensioners; – households in Prague; CONCEPTS AND SCOPE – households in communities with populations of over 5,000.
3.96 In India, CPI compilation originated from a need to maintain the purchasing power of workers’ incomes, and so four diﬀerent CPIs are compiled at the national level for reference households headed by the following kinds of workers: – agricultural labourers; – industrial workers; – rural labourers; – urban non-manual employees.

Geographical coverage 3.97 Urban and rural.

Geographical coverage may refer either to the geographical coverage of expenditures or the coverage of price collection.
Ideally these two should coincide, whether the CPI is intended to be a national or a regional index.

In most countries, prices are collected in urban areas only since their movements are considered to be representative of the price movements in rural areas.

In these cases national weights are applied and the resulting index can be considered a national CPI.
If price movements in urban and rural areas are felt to be suﬃciently diﬀerent – although price collection is restric- ted to urban areas because of resource constraints – then urban weights should be applied and the resulting index must be considered as purely an urban and not a national CPI.

For example, the following countries cover urban households only (expenditure weights and prices): Aus- tralia, Mexico, Republic of Korea, Turkey, United States.
Most other developed countries tend to use weights cov- ering urban and rural households, although in nearly every case price collection takes place in urban areas only.

Of course, the borderline between urban and rural is inevi- tably arbitrary and may vary from country to country.

For example, in France urban price collection is inter- preted to include villages with as few as 2,000 residents.
3.98 Decisions about geographical coverage in terms of urban versus rural coverage will depend on popula- tion distribution and the extent to which expenditure patterns and the movements of prices tend to diﬀer between urban and rural areas.

3.99 Foreign purchases of residents and domestic purchases of non-residents.

Problems arise when house- holds make expenditures outside the boundaries of the area or country in which they are resident.
Decisions about the treatment of such expenditures depend on the main use of a CPI.

For inﬂation analysis, it is the price change within a country which is of interest.

An index of inﬂation is needed that covers all so-called ‘‘domestic’’ consumption expenditures that take place within the geographical boundaries of the country, whether made by residents or non-residents.

HICPs (see Annex 1) are deﬁned in this way as indices of domestic inﬂation.
Thus they exclude consumption expenditures made by resi- dents when they are outside the country (which belong to the inﬂation indices of the countries where the pur- chases are made), and they include expenditures within the country made by residents of other countries.
In practice, expenditures by visitors from abroad may be 49 CONSUMER PRICE INDEX MANUAL: THEORY AND PRACTICE diﬃcult to estimate, since household budget surveys do not cover non-resident households, although estimates might be possible for some commodities using retail sales data or special surveys of visitors.

These issues become more important when there is signiﬁcant cross- border shopping as well as tourism.
3.100 When CPIs are used for escalating the incomes of residents, it may be appropriate to adopt the so-called ‘‘national’’ concept of expenditure which covers all the expenditures of residents, whether inside or outside the country, including remote purchase from non-resident outlets, for example by the Internet, tele- phone or mail.

Household budget surveys can cover all these types of expenditure, although it may be diﬃcult to identify the country from which remote (Internet, mail, etc.)
purchases are being made.

The prices paid for airline tickets and package holidays bought within the domestic territory should also be covered.

It can be diﬃcult, however, to obtain price data for the goods and services purchased by residents when abroad, although in some cases sub-indices of the partner countries’ CPIs might be used.

3.101 Regional indices.

When compiling regional indices, the concept of residence applies to the region in which a household is resident.
It is then possible to draw a distinction between the expenditures within a region and the expenditures of the residents of that region, analogous to the distinction between the ‘‘domestic’’ and ‘‘national’’ concepts of expenditure at the national level.

The same issues arise for regional indices as were dis- cussed in paragraph 3.97.

The principles applying to cross-border shopping between regions are the same as for international cross-border shopping, but data avail- ability is generally diﬀerent.
If the scope of the regional index is deﬁned to include the purchases by regional residents when in other regions (abroad), then, although price data for the other regions should be readily avail- able, it is unlikely that expenditure data will be available with the necessary split between expenditure within and expenditure outside the region of residence.

3.102 Care must be taken to treat cross-border shopping in the same way in all regions.
Otherwise double counting, or omission, of expenditures may occur when regional data are aggregated.

Where regional indices are aggregated to give a national index, the weights should be based on regional expenditure data rather than on population data alone.

3.103 Many countries try to satisfy the diﬀering needs of their many CPI users by deriving a family of indices with diﬀering coverage, headed by a single wide- ranging oﬃcial (headline) CPI which is relevant to the country as a whole.
In some large countries, regional indices are more widely used than the national CPI, particularly where the indices are used for escalating incomes.
Thus, in addition to the headline CPI, which has the widest coverage possible, subsidiary indices are published which may relate to: – sub-sectors of the population; – geographical regions; – speciﬁc commodity groups; sub-indices of the overall (oﬃcial all-items) CPI should be published at as 50 detailed a level as possible, since many users are inter- ested in the price change of speciﬁc commodity groups.
3.104 In eﬀect, many statistical oﬃces are moving towards a situation in which a database of prices and weights is maintained from which a variety of subsidiary indices is derived.

Outlet coverage 3.105 The coverage of outlets is dictated by the purchasing behaviour of the reference households.

As already stated, in principle, the prices relevant to CPIs are the prices paid by households.
In practice, however, it is usually not feasible to collect price information directly from households, although as more sales are made through electronic points of sale which record and print out both the items purchased and their prices, it may become increasingly practical to collect information on the actual transaction prices paid by households.

In the meantime, it is necessary to rely mainly on the prices at which products are oﬀered for sale in retail shops or other outlets.
All the outlets from which the reference population makes purchases are within the scope of the CPI, and should be included in the sampling frame from which the outlets are selected.
3.106 Examples of outlets are: – retail shops – from very small permanent stalls to multinational chains of stores; – market stalls and street vendors; – establishments providing household services – electricians, plumbers, window cleaners, and so on; – leisure and entertainment providers; – health and education services providers; – mail or telephone order agencies; – the Internet; – public utilities; – government agencies and departments.
3.107 The principles governing the selection of a sample of outlets from which to collect prices are dis- cussed in some detail in Chapters 5 and 6.

Price variation 3.108 Price variation occurs when exactly the same good or service is sold at diﬀerent prices at the same moment of time.

Diﬀerent outlets may sell exactly the same product at diﬀerent prices, or the same product may be sold from a single outlet to diﬀerent categories of purchasers at diﬀerent prices.
3.109 If markets were ‘‘perfect’’ in an economic sense, identical products would all sell at the same price.

If more than one price were quoted, all purchases would be made at the lowest price.

This suggests that products sold at diﬀerent prices cannot be identical but must be qualitatively diﬀerent in some way.

When the price diﬀerences are, in fact, attributable to quality diﬀer- ences, the price diﬀerences are only apparent, not genu- ine.
In such cases, a change in the average price resulting from a shift in the pattern of quantities sold at diﬀerent prices would reﬂect a change in the average quality of the products sold.

This would aﬀect the volume and not the price index.

collected from sellers and not from households, such price changes are unlikely to be recorded.
CONCEPTS AND SCOPE If 3.110 statistical oﬃces do not have enough information about the characteristics of goods and ser- vices selling at diﬀerent prices, they have to decide whether to assume that the observed price diﬀerences are genuine or only apparent.

The default procedure most commonly adopted in these circumstances is to assume that the price diﬀerences are apparent.

This assumption is typically made for both CPI and national accounts purposes.

3.111 However, markets are seldom perfect.


One reason for the co-existence of diﬀerent prices for identical products may be that the sellers are able to practise price discrimination.

Another reason may simply be that consumers lack information and may buy at higher pri- ces out of ignorance.

Also, markets may be temporarily out of equilibrium as a result of shocks or the appearance of new products.

It must be recognized, therefore, that genuine price diﬀerences do occur.
Price discrimination 3.112 Economic theory shows that price discrimi- nation tends to increase proﬁts.

It may not be feasible to practise price discrimination for goods because they can be retraded.

Purchasers discriminated against would not buy directly but would try to persuade those who could purchase at the lowest prices to buy on their behalf.

Services, however, cannot be retraded, as no exchange of ownership takes place.
3.113 Price discrimination appears to be extremely common, almost the norm, for many kinds of services including health, education and transport.

For example, senior citizens may be charged less than others for exactly the same kinds of health or transportation ser- vices.

Universities may charge foreign students higher fees than domestic students.
As it is also easy to vary the qualities of the services provided to diﬀerent consumers, it can be diﬃcult to determine to what extent observed price diﬀerences are a result of quality diﬀerences or pure price discrimination.

Sellers may even attach trivial or spurious diﬀerences in terms or conditions of sale to the services sold to diﬀerent categories of purchasers in order to disguise the price discrimination.

3.114 Price discrimination can cause problems with regard to price indices.
Suppose, for example, that a service supplier discriminates by age by charging senior citizens aged 60 years or over price p2 and others price p1, where p1>p2.

Suppose, further, that the supplier then decides to redeﬁne senior citizens as those aged 70 years or over while otherwise keeping prices unchanged.

In this case, although neither p1 nor p2 changes, the price paid by individuals aged 60 to 70 years changes and the average price paid by all households increases.
3.115 This example illustrates a point of principle.

Although neither of the stated prices, p1 and p2, at which the services are on oﬀer changes, the prices paid by certain households do change if they are obliged to switch from p2 to p1.

From the perspective of the households, price changes have occurred and a CPI should, in principle, record a change.

When prices are Price variation between outlets 3.116 The existence of diﬀerent prices in diﬀerent outlets raises similar issues.
Pure price diﬀerences are almost bound to occur when there are market imper- fections, if only because households are not perfectly informed.

When new outlets open selling at lower prices than existing ones, there may be a time lag during which exactly the same item sells at diﬀerent prices in diﬀerent outlets because of consumer ignorance or inertia.
3.117 Households may choose to switch their pur- chases from one outlet to another or even be obliged to switch because the universe of outlets is continually changing, some outlets closing down while new outlets open up.

When households switch, the eﬀect on the CPI depends on whether the price diﬀerences are pure or apparent.

When the price diﬀerences are genuine, a switch between outlets changes the average prices paid by households.

Such price changes ought to be captured by CPIs.


On the other hand, if the price diﬀerences reﬂected quality diﬀerences, a switch would change the average quality of the products purchased, and hence aﬀect volume, not price.

3.118 Most of the prices collected for CPI purposes are oﬀer prices and not the actual transactions prices paid by households.

In these circumstances, the eﬀects of switches in the pattern of households’ purchases between outlets may remain unobserved in practice.
When the price diﬀerences reﬂect quality diﬀerences, the failure to detect such switches does not introduce any bias into the CPI.

Buying at a lower price means buying a lower-quality product, which does not aﬀect the price index.

However, when the price diﬀerences are genuine, the failure to detect switches will tend to introduce an upward bias in the index, assuming households tend to switch towards outlets selling at lower prices.

This potential bias is described as outlet substitution bias.
Outlet rotation 3.119 A further complication is that, in practice, prices are collected from only a sample of outlets and the samples may change, either because outlets open and close or because there is a deliberate rotation of the sample periodically.

When the prices in the outlets newly included in the sample are diﬀerent from those in the previous outlets, it is again necessary to decide whether the price diﬀerences are apparent or genuine.
If they are assumed to be apparent, the diﬀerence between the price recorded previously in an old outlet and the new price in the new outlet is not treated as a price change for CPI purposes, the diﬀerence being treated as attributable to quality diﬀerence.

As explained in more detail in Chapter 7, if this assumption is correct, the price changes recorded in the new outlets can simply be linked to those previously recorded in the old outlets without intro- ducing any bias into the index.
The switch from the old to the new outlets does not have any impact on the CPI.

If the price diﬀerences between the old and the new outlets are deemed to be genuine, however, the 3.120 51 CONSUMER PRICE INDEX MANUAL: THEORY AND PRACTICE simple linking just described can lead to bias.

When households change the price they pay for a product by changing outlets, the price changes should be captured by the CPI.
As explained in more detail in Chapter 7, it seems that most statistical oﬃces tend to assume that the price diﬀerences are not genuine and simply link the new price series on to the old.

Given that it is un- realistic to assume that markets are always perfect and that pure price variation never occurs, this procedure, although widely used, is questionable and may lead to upward bias.

Such bias is described as outlet rotation bias.


One possible strategy that has been suggested is to assume that half of any observed price diﬀerence between old and new outlets is genuine and half is a result of quality diﬀerence, on the grounds that, although inevitably somewhat arbitrary, it is likely to be closer to the truth than assuming that the diﬀerence is either entirely genuine or entirely attributable to quality dif- ferences (see McCracken, Tobin et al., 1999).
Treatment of some speciﬁc household expenditures 3.121 Some of the expenditures made by households may not be on goods and services for household con- sumption and may therefore fall outside the scope of a CPI.

One major category consists of the business expen- ditures made by households.
Fees of agents and brokers 3.122 When a house is purchased for own use by an owner-occupier, it can be argued that the transfer costs associated with purchase (and sale) should be treated as consumption expenditures in the same way as the bro- kers’ fees incurred when ﬁnancial assets are bought or sold.

The fees paid to an agent to buy or sell houses are included in many national CPIs, provided that the house is to be occupied by the owner and not rented to a third party.
Undesirable or illegal goods and services 3.123 All the goods and services that households willingly purchase in order to satisfy their personal needs or wants constitute consumers’ expenditures and there- fore fall within the scope of a CPI, irrespective of whe- ther their production, distribution or consumption is illegal or carried out in the underground economy or on the black market.
Particular kinds of goods or services must not be excluded because they are considered to be undesirable, harmful or objectionable.

Such exclusions could be quite arbitrary and undermine the objectivity and credibility of the CPI: First, it should be noted that some goods and services might be deemed to be undesirable at some times and desirable at others, or vice versa.

People’s attitudes change as they acquire more information, especially as a result of scientiﬁc advances.
Similarly, some goods or services may be deemed to be undesirable in some countries but not in others at the same point of 52 time.

The concept of an undesirable good is inher- ently subjective and somewhat arbitrary and volatile.

 Second, if it is accepted that some goods and services may be excluded on the grounds that they are un- desirable, the index is thereby exposed to actual or attempted manipulation by pressure groups.
Third, attempts to exclude certain goods or services by pressure groups may be based on a misunderstanding of the implications of so doing.

For example, if the CPI is used for escalating incomes, it may be felt that households ought not be compensated for increases in the prices of certain undesirable products.

However, excluding them does not imply lowering the index.
A priori, excluding some item is just as likely to increase the CPI as reduce it, depending on whether the price increase for the item in question is below or above the average for other goods and services.

For example, if it is decided to exclude smoking from a CPI and the price increase for smoking products is below average, excluding smoking actually increases the income of smokers (just as it does for non-smokers).
3.124 While goods and services that households willingly choose to consume should not, in principle, be excluded from a CPI because they are acquired in the underground economy or even illegally, it may be impos- sible to obtain the requisite data on the expenditures or the prices, especially on illegal goods and services.

They may well be excluded in practice.


Luxury goods and services 3.125 When a CPI is used as an index of general inﬂation, it ought to include all households regardless of their socio-economic group and also all consumer goods and services regardless of how expensive they are.
Simi- larly, the scope of an index used for purposes of esca- lating incomes should include all the goods and services purchased by the reference households, irrespective of whether any of these goods and services are considered to be luxuries or otherwise unnecessary or undesirable.

3.126 Of course, if the reference households are conﬁned to a select group of households, the index will eﬀectively exclude all those items that are purchased exclusively by households that are not in the group.
For example, excluding the wealthiest 5 per cent of house- holds will, in practice, exclude many luxury items from the scope of the index.

As already noted, such house- holds may be excluded for various reasons, including the unreliability of their expenditure data and the fact that collecting prices for some items purchased exclu- sively by a tiny minority of households may not be cost- eﬀective.
Once the group of reference households has been decided and deﬁned, however, judgements should not be made about whether to exclude certain of their expenditures that are considered to be non-essential or on luxuries.

Second-hand goods 3.127 Markets for used or second-hand goods exist for most durable goods.

Household expenditures include expenditures on second-hand goods and are therefore within the scope of a CPI.
Households’ sales of durables constitute negative expenditures, however, so that the weights for second-hand goods are based on households’ net expenditures: i.e., total purchases less sales.

The total expenditure on a particular type of second-hand good is a function of the rate at which it is bought and sold, i.e., a higher turnover rate (number of transactions) gives a higher total expenditure.
A higher turnover does not, however, increase the rate at which any individual good can be used for purposes of consumption or the ﬂow of services that may be obtained from the good.

3.128 Households may buy second-hand goods through any of the following routes: Directly from another household – the selling house- hold will record the proceeds of the sale as receipts.
Net expenditures, i.e., expenditures less receipts, are zero so no weight is attached to purchases and sales from one household to another.

 From another household via a dealer – in principle, households’ expenditures on the services of the dealers are given by the values of their margins (the diﬀerence between their buying and selling prices).

These inter- mediation services should be included in CPIs.
They should be treated in the same way as the fees charged by agents such as ﬁnancial auxiliaries.

The margins may be extremely diﬃcult to estimate in practice.

Care should be taken to include trade-ins either as pur- chases by the dealers or receipts of households.

 Directly from another sector, i.e., from an enterprise or from abroad – the weight would be household pur- chases of the second-hand goods from other sectors less sales to other sectors.
From an enterprise or from abroad via a dealer – the appropriate weight is given by household purchases from dealers less any household sales to dealers plus the aggregate of dealers’ margins on the products that they buy from and resell to households.

Trade-ins should count as part of sales by households (in the case of cars, the weight given to new cars should not include any deduction for the value of trade-ins).
3.129 In some countries, many of the durables pur- chased by households, especially vehicles, may be imports of second-hand goods from other countries.

The prices and expenditures on these goods enter the CPI in the same way as those for newly produced goods.

Similarly, in some countries there may be signiﬁcant net pur- chases of second-hand vehicles by households from the business sector, these vehicles possibly carrying more weight in the index than new vehicles purchased by households.
Imputed expenditures on goods and services 3.130 As explained in earlier sections, many of the goods and services acquired and used by households for purposes of their own ﬁnal consumption are not pur- chased in monetary transactions but are acquired through barter or as remuneration in kind or are pro- duced by households themselves.
It is possible to esti- mate what households would have paid if they had CONCEPTS AND SCOPE purchased these goods and services in monetary trans- actions or, alternatively, what it cost to produce them.

In other words, values may be imputed for these non- monetary expenditures.

3.131 The extent to which it is desirable to include imputed expenditures within the scope of a CPI depends partly on the main purpose of the index.
If the CPI is intended to be a measure of consumer inﬂation, it can be argued that only monetary expenditures should be included.

Inﬂation is a monetary phenomenon measured by changes in monetary prices recorded in monetary transactions.

Even when the main use of a CPI is for indexation purposes, it can be argued that it should only reﬂect changes in the monetary prices actually paid by the reference population.
Consistent with the objective of monitoring inﬂation in the European Union, the aim of the Harmonized Index of Consumer Prices (HICP) compiled by Eurostat is to measure inﬂation faced by consumers.

The concept of ‘‘household ﬁnal monetary consumption expenditure’’ (HFMCE) used in the HICP deﬁnes both the goods and services to be covered, and the price concept to be used, i.e., prices net of reimburse- ments, subsidies and discounts.
HFMCE refers only to monetary transactions and includes neither consumption of own production (e.g., agricultural goods or owner- occupied housing services) nor consumption of goods and services received as income in kind.

3.132 When the CPI is intended to be a cost of living index, some imputed expenditures would normally be included within the scope of the CPI on the grounds that the goods and services acquired in non-monetary trans- actions aﬀect households’ living standards.
As already imputed noted, most countries include households’ expenditures on housing services produced by owner- occupiers but not imputed expenditures on goods such as agricultural goods produced for own consumption.

Price coverage 3.133 A CPI should reﬂect the experience of the consumers to whom it relates, and should therefore record what consumers actually pay for the goods and services which are included in the scope of the index.
The expenditures and prices recorded should be those paid by consumers, including any taxes on the products, and taking account of all discounts, subsidies and most rebates, even if discriminatory or conditional.

It may be virtually impossible, however, to take account of all discounts and rebates in practice.

Sensible practical compromises are needed, for which recommendations and examples are given in Chapter 6.
3.134 When households pay the full market prices for products and are then subsequently reimbursed by governments or social security schemes for some of the amounts paid, CPIs should record the market prices less the amounts reimbursed.

This kind of arrangement is common for educational and medical expenditures.
Taxes and subsidies 3.135 All taxes on products, such as sales taxes, excise taxes and value added tax (VAT), are part of 53 CONSUMER PRICE INDEX MANUAL: THEORY AND PRACTICE the purchasers’ prices paid by consumers that should be used for CPI purposes.

Similarly, subsidies should be taken into account, being treated as negative taxes on products.
3.136 For some analytical and policy purposes, it may be useful to estimate a CPI that measures price movements excluding the eﬀects of changes in taxes and subsidies.

For monetary policy-makers, the price increases resulting from changes in indirect taxes or subsidies are not part of an underlying inﬂationary process but are attributable to their own manipulation of these economic levers.
Similarly, when a CPI is used for escalation purposes, any increase in a CPI resulting from increases in indirect taxes leads to an increase in wages and beneﬁts linked to the CPI, despite the fact that the aim of the tax increase might have been to reduce con- sumers’ purchasing power.

Alternatively, an increase in subsidies might be intended to stimulate consumption, but the resulting lower prices could be oﬀset by a smaller increase in indexed wages and beneﬁts.

3.137 Net price indices.


Net price indices may be compiled in which taxes on consumer goods or services are deducted from the purchasers’ prices, and subsidies are added back on.

Such indices do not, however, necessarily show how prices would have moved if there were no taxes or changes in taxes.

It is notoriously dif- ﬁcult to estimate the true incidence of taxes on products: that is, the extent to which taxes or subsidies, or changes therein, are passed on to consumers.
It is also diﬃcult to take account of the secondary eﬀects of changes in taxes.

In order to estimate the secondary eﬀects, input–output analysis can be used to work out the cumulative impact of taxes and subsidies through all the various stages of production.
For example, some of the taxes on vehicle fuel will enter the price of transport services which in turn will enter the prices of transported goods, some of which will enter the prices paid for consumer goods by retailers and hence the prices which they charge to con- sumers.

To track all these impacts would demand a much more detailed and up-to-date input–output table than is available in most countries.
A more practicable alter- native is therefore simply to conﬁne the taxes and sub- sidies for which correction is made to those levied at the ﬁnal stage of sale at retail; that is, primarily to VAT, sales and excise taxes.

Estimating prices less these taxes only, or corrected for changes in these taxes only, is more feasible.
In the case of a percentage sales tax or VAT, the calculation is simple, but in the case of excise taxes, it is necessary to ascertain the percentage mark-up by the retailer, since the excise tax will also be marked up by this percentage.

Discounts, rebates, loyalty schemes and ‘‘free’’ products 3.138 CPIs should take into account the eﬀects of rebates, loyalty schemes, and money-oﬀ vouchers.
Given that a CPI is meant to cover all the reference households, whether in the country as a whole or in a particular region, discounts should be included even if they are available only to certain households or to consumers satisfying certain payment criteria.

54 3.139 It may be diﬃcult to record discriminatory or conditional discounts for practical reasons.
When only one selected group of households can enjoy a certain discount on a speciﬁc product, the original stratum for that product is split into two new strata, each experiencing diﬀerent price changes and each requiring a weight.

So, unless base period expenditures for all possible strata are known, it is not possible to record discriminatory discounts correctly.

Similarly, with con- ditional discounts, e.g.


discounts on utility bills for prompt payment, it can be diﬃcult to record the eﬀect of the introduction of such oﬀers unless data are available on the proportion of customers taking advantage of the oﬀer.
These kinds of practical problems also arise when there is price discrimination and the sellers change the criteria that deﬁne the groups to whom diﬀerent prices are charged, thereby obliging some households to pay more or less than before without changing the prices themselves.

These cases are discussed further in Chapter 7.


3.140 Although it is desirable to record all price changes, it is also important to ensure that the qualities of the goods or services for which prices are collected do not change in the process.

While discounted prices may be collected during general sales seasons, care should be taken to ensure that the quality of the products being priced has not deteriorated.
3.141 The borderline between discounts and rebates can be hazy and is perhaps best drawn according to timing.

In other words, a discount takes eﬀect at the time of purchase, whereas a rebate takes eﬀect some time later.

Under this classiﬁcation, money-oﬀ vouchers are discounts, and as with the conditional discounts men- tioned above, can only be taken into account in a CPI if they relate to a single product and if the take-up rate is known at the time of CPI compilation.
Since this is highly unlikely, the eﬀect of money-oﬀ vouchers is usually excluded from a CPI.

It should be noted that the discount is recorded only when the voucher is used, not when the voucher is ﬁrst made available to the con- sumer.

3.142 Rebates may be made in respect of a single product, e.g.

air miles, or may be more general, e.g.

supermarket loyalty schemes where a $10 voucher is awarded for every $200 spent.
As with discounts dis- cussed above, such rebates can only be recorded as price falls if they relate to single products and can be weighted according to take-up.

Bonus products provided ‘‘free’’ to the consumer, either by larger pack sizes or oﬀers such as ‘‘two packs for the price of one’’, should be treated as price reductions, although they may be ignored in practice when the oﬀers are only temporary and quickly reversed.
When permanent changes to pack sizes occur, quality adjustments should be made (see Chapter 7).

3.143 Given the practical diﬃculties in correctly recording all these types of price falls, it is usual to reﬂect discounts and rebates only if unconditional, whereas loyalty schemes, money-oﬀ coupons, and other incentives are ignored.

Discounts during seasonal sales may be recorded provided that the quality of the goods does not change.
Classiﬁcation 3.144 The classiﬁcation system upon which any CPI is built provides the structure essential for many stages of CPI compilation.

Most obviously, it provides the weighting and aggregation structure, but it also provides the scheme for stratiﬁcation of products in the sampling frame, at least down to a certain level of detail, and it dictates the range of sub-indices available for publica- tion.

Several factors must be taken into account when a CPI classiﬁcation system is being developed.
First, the classiﬁcation must reﬂect economic reality.

For example, it must be possible to accommodate new goods and services in a manner that minimizes the need for later restructuring of the higher-level cate- gories.

Restructuring is undesirable because many users require long time series, and restructuring of the classiﬁcation will produce breaks in the series.
Second, the needs of users for sub-indices should be given a high priority when constructing aggregate groups, so that if, for example, some users are parti- cularly interested in price change in food products, then the classiﬁcation should provide suﬃcient detail in that area.

 Third, it is a requirement of any classiﬁcation that its categories are unambiguously mutually exclusive, and at the same time provide complete coverage of all products considered to be within its scope.
In practice this means that it should be a straightforward task to assign any particular expenditure, or price, to a single category of the classiﬁcation system.

3.145 The availability and nature of the data them- selves will also aﬀect the design of a classiﬁcation system.

The availability of expenditure and price data will dictate the lowest that might be possible.

Obviously it is not possible to produce a separate index for a product for which either weights or prices are not available.
At the most detailed level, a high variance of the price changes, or relatives, will suggest where additional categories are needed.

In line with standard sampling procedures, the stratiﬁcation scheme should minimize the within-stratum variance while at the same time maximizing the between-stratum variance.

The classiﬁcation should reﬂect this requirement.


level of detail Criteria for classifying consumption expenditure 3.146 Although a classiﬁcation may be conceived according to economic theory or user requirements using a top-down approach, in practice the statistical compiler collects data about individual products and then aggre- gates them according to the classiﬁcation scheme (a bottom-up application).
For example, the units of clas- siﬁcation for the Classiﬁcation of Individual Consump- tion according to Purpose (COICOP) are expenditures for the acquisition of consumer goods and services, not expenditures on purposes as such.
Divisions 01 to 12 of COICOP convert these basic statistics into a purpose classiﬁcation by grouping together the various goods and services which are deemed to fulﬁl particular pur- CONCEPTS AND SCOPE poses, such as nourishing the body, protecting it against inclement weather, preventing and curing illness, acquiring knowledge, travelling from one place to another, and so on.
3.147 Classiﬁcations of expenditure data are schemes for aggregating expenditures on products according to certain theoretical or user-deﬁned criteria, such as: Product type – products may be aggregated by: – physical characteristics of goods and the nature of services; for example, biscuits are divided into those with and without a chocolate coat.
This cri- terion can be meaningfully implemented down to the most detailed level, and is the basis of the Central Product Classiﬁcation 1.0 (United Nations, 1998b); – economic from which the product originated.
The International Standard Industrial Classiﬁcation of All Economic Activities (ISIC), Revision 3.1 (United Nations, 2002) is the inter- national standard classiﬁcation; activity – production process from which the product origi- nated; – retail outlet type from which the product was pur- chased; – geographical origin of the product.

 Purpose to which the products are put, e.g.

to provide food, shelter, transport, etc.

COICOP is the interna- tional standard.


 The economic environment, where products could be aggregated according to criteria such as: – substitutability of products; – complementarity of products; – application of sales taxes, consumer subsidies, excise taxes, customs duties, etc.

; – imports from diﬀerent countries (and in some cases, a classiﬁcation of exportable products may be of interest).
Classiﬁcation by product type 3.148 Where indices of price change for speciﬁc products groups are required, a product-based classiﬁ- cation would be appropriate.

Product classiﬁcations may combine several of the criteria listed above; for example, the Classiﬁcation of Products by Activity (CPA) in the European Economic Community (Eurostat, 1993), which is linked to the CPC at the detailed level and the ISIC at the aggregate level.
3.149 Inevitably, price collectors and index compil- ers will encounter products for which no detailed class or sub-class exists, for example, entirely new products, or mixed products which are bundles of existing products.

This is a problem frequently encountered with high technology goods, telecommunications goods and ser- vices, and food items in the form of ‘‘ready meals’’.

Initially, the expenditure on these products may be recorded in an ‘‘other’’ or n.e.c.
(not elsewhere classiﬁed) class, but once expenditure on these products becomes signiﬁcant, a separate class should be created.

55 CONSUMER PRICE INDEX MANUAL: THEORY AND PRACTICE Classiﬁcation by purpose 3.150 For a CPI compiler aiming to produce a measure of the change in the cost of satisfying particular needs, a purpose-based classiﬁcation is appropriate.
The COICOP breakdown at the highest level is by purpose such that the 12 divisions of COICOP are categories of purpose, and below this level the groups and classes are product types.

In other words, products are allocated to purpose headings.
The allocation of products is com- plicated by the existence of multi-purpose products (single products that can be used for a variety of pur- poses), such as electricity, and mixed purpose (bundled) products, such as package holidays comprising trans- port, accommodation, meals, and so on.

3.151 Multi-purpose goods and services.

The majority of goods and services can be unambiguously assigned to a single purpose, but some goods and services could plausibly be assigned to more than one purpose.
Exam- ples include motor fuel, which may be used to power vehicles classiﬁed as transport as well as vehicles classi- ﬁed as recreational, and snowmobiles and bicycles which may be bought for transport or for recreation.

3.152 In drawing up COICOP, the general rule fol- lowed has been to assign multi-purpose goods and ser- vices to the division that represents the predominant purpose.

Hence, motor fuel is shown under ‘‘Transport’’.


Where the predominant purpose varies between coun- tries, multi-purpose items have been assigned to the division that represents the main purpose in the countries where the item concerned is particularly important.
As a result, snowmobiles and bicycles are both assigned to ‘‘Transport’’ because this is their usual function in the regions where most of these devices are purchased – that is, North America and the Nordic countries in the case of snowmobiles, and Africa, South-East Asia, China and the low countries of Northern Europe in the case of bicycles.
3.153 Examples of other multi-purpose items in COICOP include: food consumed outside the home, which is shown under ‘‘Hotels and restaurants’’, not ‘‘Food and non-alcoholic beverages’’; camper vans, which are shown under ‘‘Recreation and culture’’, not ‘‘Transport’’; and basket-ball shoes and other sports footwear suitable for everyday or leisure wear, which are shown under ‘‘Clothing and footwear’’, not ‘‘Recreation and culture’’.
3.154 National statisticians may wish to reclassify multi-purpose items if they consider that an alternative purpose is more appropriate in their country.

Such reclassiﬁcations should be footnoted.

3.155 Mixed purpose goods and services.

Single out- lays may sometimes comprise a bundle of goods and services which serve two or more diﬀerent purposes.
For example, the purchase of an all-inclusive package tour include payments for transport, accommodation will and catering services, while the purchase of educational services may include payments for health care, trans- port, accommodation, board, educational materials, and so on.

3.156 Outlays covering two or more purposes are dealt with case by case with the aim of obtaining a 56 breakdown by purpose that is as precise as possible and consistent with practical considerations of data avail- ability.
Hence, purchases for package holidays are shown under ‘‘Package holidays’’ with no attempt to isolate separate purposes such as transport, accommodation and catering.

Payments for educational services, in contrast, should as far as possible be allocated to ‘‘Education’’, ‘‘Health’’, ‘‘Transport’’, ‘‘Hotels and restaurants’’ and ‘‘Recreation and culture’’.
3.157 Two other examples of mixed purpose items are: the purchase of in-patient hospital services which include payments for medical treatment, accommoda- tion and catering; and the purchase of transport services which include meals and accommodation in the ticket price.

In both cases, there is no attempt to isolate separate purposes.
Purchases of in-patient hospital ser- vices are shown under ‘‘Hospital services’’ and pur- chases of transport services with accommodation and catering are shown under ‘‘Transport services’’.

Classiﬁcations for consumer price indices 3.158 In practice, most countries use a hybrid clas- siﬁcation system for their CPI in the sense that the breakdown of expenditure at the highest level is by purpose, with breakdowns by product at the lower levels.
In some countries the higher-level purpose clas- siﬁcations were developed many years ago for CPIs that were originally devised as measures of the changing cost of a basket of goods and services that were, at the time, considered necessary for survival or maintaining some ‘‘basic’’ standard of living.

Thus, the classiﬁcations were based on consumer needs, where ‘‘need’’ may have had a somewhat subjective interpretation depending on po- litical requirements.
3.159 The recommended practice today is still to use a purpose classiﬁcation at the highest level, with product breakdowns below, but to use the recently developed international standard classiﬁcations as far as possible, with adaptations to national requirements where necessary.

In other words, divisions 01 to 12 of COICOP, with Central Product Classiﬁcation (CPC) product classes and sub-classes mapped onto them to provide the next two levels of detail.
Publication level 3.160 As mentioned above, any restructuring of the classiﬁcation of published indices will inconvenience users and should be avoided so far as possible by careful planning and development of the classiﬁcation scheme in the ﬁrst place.

There is a trade-oﬀ between providing users with as much detail as they would like in terms of product indices and weights, and preserving some free- dom to restructure the lower levels (unpublished) with- out apparently aﬀecting the published series.
3.161 Item samples below the level at which weights are published can be revised between major weight revisions.

As explained in Chapter 9, new and replace- ment items and varieties can also be introduced pro- vided they can be included within an existing published weight.
A major new product, such as a personal computer, could only be introduced at the time of a major weight revision, whereas it might be possible to introduce mobile phones at any time if the lowest-level weight published in the telecommunications category is for telephone services.

Classiﬁcation of Individual Consumption according to Purpose (COICOP) 3.162 COICOP structure.
The international stan- dard classiﬁcation of individual consumption expendi- tures is the Classiﬁcation of Individual Consumption according to Purpose (COICOP).

COICOP is a func- tional classiﬁcation that is also used in SNA 1993 and covers the individual consumption expenditures in- curred by three institutional sectors, i.e.

households, non-proﬁt institutions serving households (NPISHs), and general government.

consumption expenditures are those which beneﬁt individual persons or households.
Individual 3.163 COICOP has 14 divisions: – divisions 01 to 12 covering the ﬁnal consumption expenditure of households; – division 13 covering the ﬁnal consumption expendi- ture of NPISHs; – division 14 covering the individual consumption ex- penditure of general government.

The classiﬁcation has three levels of detail: – division or two-digit level, e.g.

01.

Food and non-alco- holic beverages; – group or three-digit level, e.g.

01.1 Food; – class or four-digit level, e.g.

01.1.1 Bread and cereals.


3.164 The 12 divisions covering households consist of 47 groups and 117 classes and are listed in Annex 2.

Below the level of class, CPI compilers have to create additional detail by further subdividing the classes according to their national needs.
Of course, there are clear advantages, in terms of comparability between countries, and between the diﬀerent uses of COICOP (CPIs, household expenditure statistics, national ac- counts aggregates), if the basic, higher-level structure of COICOP is maintained.

3.165 There are some COICOP classes which may, or may not, be included in most CPIs, or for which expenditure data cannot be collected directly from households.
For example, COICOP has a class for the imputed rentals of owner-occupiers, which may be outside the scope of some CPIs.

COICOP also has a intermediation services indirectly class for ﬁnancial measured, which may be outside the scope of some CPIs because of practical measurement diﬃculties.

In any case, the expenditures on these services cannot be col- lected in household budget surveys.
Similarly, COICOP has a group for expenditure on insurance service CONCEPTS AND SCOPE charges, which may be within the scope of CPIs but cannot be measured using household surveys.

3.166 Type of product.

COICOP classes are divided into: services (S), non-durables (ND), semi-durables (SD) and durables (D).

This supplementary classiﬁcation provides for other analytical applications.
For example, an estimate may be required of the stock of consumer durables held by households, in which case the goods in COICOP classes that are identiﬁed as ‘‘durables’’ pro- vide the basic elements for such estimates.

3.167 As explained above, the distinction between non-durable goods and durable goods is based on whe- ther the goods can be used only once or whether they can be used repeatedly or continuously over a period of considerably more than one year.
Moreover, durables, such as motor cars, refrigerators, washing machines and televisions, have a relatively high purchasers’ value.

Semi-durable goods diﬀer from durable goods in that their expected lifetime of use, though more than one year, is often signiﬁcantly shorter and their purchasers’ value is substantially less.
Because of the importance attached to durables, the categories of goods deﬁned as durables in COICOP are listed below: – furniture, furnishings, carpets and other ﬂoor coverings; – major household appliances; – tools and equipment for house and garden; – therapeutic appliances and equipment; – vehicles; – telephone and fax equipment; – audiovisual, photographic and information pro- cessing equipment (except recording media); – major durables for recreation; – electrical appliances for personal care; – jewellery, clocks and watches.
The following goods are listed as semi-durables: – clothing and footwear; – household textiles; – small electrical household appliances; – glassware, tableware and household utensils; – spare parts for vehicles; – recording media; – games, toys, hobbies, equipment for sport, camping, etc.

3.168 Some COICOP classes contain both goods and services because it is diﬃcult for practical reasons to break them down into goods and services.
Such classes are usually assigned an (S) when the service component is considered to be predominant.

Similarly, there are classes which contain either both non-durable and semi- durable goods or both semi-durable and durable goods.

Again, such classes are assigned a (ND), (SD) or (D) according to which type of good is considered to be the most important.

57 CONSUMER PRICE INDEX MANUAL: THEORY AND PRACTICE Appendix 3.1 Consumer price indices and national accounts price deﬂators 1.
The purpose of this appendix is to explain why and how consumer price indices (CPIs) diﬀer from the price indices used to deﬂate household consumption expenditures in national accounts.

The diﬀerences between the two kinds of price index are often not well understood.

Coverage of households 2.

The sets of households covered by CPIs and the national accounts are not intended to be the same, CPIs typically cov- ering a smaller set of households.
Household consumption expenditures in national accounts cover the expenditures made by all households, including institutional households resident in the country or region, whether those expenditures are made inside or outside the country or region of residence.

CPIs tend to cover the expenditures and prices paid by households within the geographical boundaries of a country or region, whether the households are residents or visitors.
More importantly, most CPIs are purposely deﬁned to cover only selected groups of non-residential households.

For example, CPIs may exclude very wealthy households or be conﬁned to households in urban areas or headed by wage-earners.

products or other goods produced for own consumption that are included in national accounts.

Timing 4.

Most CPIs measure price changes between two points of time or very short intervals of time such as a week.
The price indices in national accounts are intended to deﬂate expendi- tures aggregated over long periods of time, generally a year.

The ways in which monthly or quarterly CPIs are averaged to obtain annual CPI indices are unlikely to be conceptually consistent with the annual price indices in national accounts.

Index number formulae 5.

The index number formulae used by CPIs and national accounts are not intended to be the same.
In practice, most CPIs tend to use some kind of Lowe price index that uses the quantities of an earlier period, whereas the price indices, or price deﬂators, in national accounts are usually meant to be Paasche indices.

Paasche indices are used in order to obtain Laspeyres volume indices.

These diﬀerences, arising from the use of diﬀerent index formulae, would tend to be reduced if both CPIs and national accounts adopted annual chaining.

Conclusions Coverage of consumption expenditures 3.
The sets of expenditures covered by CPIs and national accounts are not intended to be the same, CPIs typically covering a smaller set of expenditures.

Most CPIs do not cover most of the imputed non-monetary consumption expenditures included in national accounts, either on principle or in practice because of lack of data.

Many CPIs include the imputed rents on owner-occupied housing, but CPIs are not intended to cover the imputed expenditures and prices of agricultural 6.
It is clear that, in general, CPIs and the price deﬂators for national accounts can diﬀer for a variety of reasons, such as major diﬀerences in the coverage of households and expendi- tures, diﬀerences in timing and diﬀerences in the underlying index number formulae.

These diﬀerences are intentional and justiﬁed.


Of course, the price data collected for CPI purposes may also be used to build up the detailed price deﬂators used for national accounts purposes, but at an aggregate level CPIs and national accounts deﬂators may be quite diﬀerent for the reasons just given.
58 EXPENDITURE WEIGHTS AND THEIR SOURCES 4 Introduction 4.1 A consumer price index (CPI) is usually calcu- lated as a weighted average of the price changes for the consumption goods and services covered by the index.

The weights are meant to reﬂect the relative importance of the goods and services as measured by their shares in the total consumption of households.

The weight attached to each good or service determines the impact that its price change will have on the overall index.
The weights should be made publicly available in the interests of transparency, and for the information of the users of the index.

4.2 The weights depend on the scope of the index which, in turn, depends on the main use, or uses, for the index.

The uses and scope of a CPI have already been explained in some detail in the two previous chapters.

This chapter therefore focuses on the derivation and compilation of the weights and the data sources that may be used to estimate them.
In practice, the weights usually refer to expenditures on consumption goods and services by households, as distinct from the actual use of those goods and services to satisfy the needs and wants of households.

Expenditure-based weights are appro- priate for a CPI based on the acquisitions approach.

The diﬀerence between the acquisitions and uses approach to CPIs was explained in the previous chapter.
4.3 In the special case of owner-occupied housing, however, many countries adopt the uses rather than the acquisitions approach.

They measure changes in the prices of the ﬂows of housing services consumed by households as distinct from changes in the prices of dwellings.

It is shown in Chapter 23 of this manual that one important consequence of adopting the uses approach to owner-occupied housing is that its weight in the overall CPI is considerably greater than when the acquisitions approach is used.
The reason is that the values of the housing services consumed by owner- occupiers have to cover not only the depreciation on the houses purchased but also the interest costs on the capital invested in the dwellings.

Over a period of years, the uses approach may well give twice as much weight to owner-occupied housing as the acquisitions approach.

Reference may be made to Chapter 23 for further details and explanation.

The weighting structure of the consumer price index each of the elementary aggregates.
Elementary indices are constructed by (a) collecting a sample of repre- sentative prices for each elementary aggregate, and then (b) calculating an average price change for the sample.

In the second stage, a weighted average is taken of the elementary indices using the expenditures within the elementary aggregates as weights.

4.5 Elementary aggregates are usually the smallest groups of goods and services for which expenditure data are available to be used as weights.
They may cover the whole country or separate regions within the country.

Likewise, elementary aggregates may be distinguished for diﬀerent types of outlets.

The nature of the ele- mentary aggregates depends on circumstances and the availability of expenditure data.

Elementary aggregates may therefore be deﬁned diﬀerently in diﬀerent coun- tries.

In general: Elementary aggregates should consist of groups of goods or services that are as similar as possible.
They should also consist of goods or services that may be expected to have similar price movements.

The objective is to minimize the dispersion of price move- ments within the aggregate.

 The elementary aggregates should be appropriate to serve as strata for sampling purposes in the light of the sampling regime planned for the data collection.
4.6 The aggregation structure for a CPI is illustrated in Figure 4.1 using the Classiﬁcation of Individual Consumption according to Purpose (COICOP) des- cribed in Chapter 3, although similar national classiﬁ- cations may be used instead: First, the entire set of consumption goods and services covered by the overall CPI is divided into divisions, such as ‘‘food and non-alcoholic beverages’’.

 Each division is then divided into groups, such as ‘‘food’’.
Each group is further divided into classes, such as ‘‘bread and cereals’’.

 Each class may be divided into more homogeneous sub-classes, such as ‘‘rice’’.

 Finally, a sub-class may be further subdivided to obtain the elementary aggregates, by dividing accord- ing to region or type of outlet, as illustrated in Figure 4.1.

In some cases, a particular sub-class cannot be, or does not need to be, further subdivided, in which case the sub-class becomes the elementary aggregate.
4.4 As explained in more detail in Chapters 7 and 9, the calculation of a CPI usually proceeds in two stages.

In the ﬁrst stage, elementary indices are estimated for The sub-classes and elementary aggregates are not part of COICOP itself but more detailed breakdowns of COICOP classes that are needed for CPI purposes.
59 CONSUMER PRICE INDEX MANUAL: THEORY AND PRACTICE Figure 4.1 Typical aggregation structure of a consumer price index (CPI) OVERALL CPI products GROUP Food and non- alcoholic beverages GROUP Alcoholic beverages and tobacco OTHER GROUPS CLASS Bread and cereals CLASS Meat OTHER CLASSES SUB-CLASS Rice SUB-CLASS Bread OTHER SUB- CLASSES Sold in northern region Sold in southern region Sold in other regions ELEMENTARY AGGREGATE Rice sold in northern supermarkets ELEMENTARY AGGREGATE Rice sold in other northern outlets REPRESENTATIVE PRODUCT Parboiled long-grain white rice REPRESENTATIVE PRODUCT Brown rice: over 50 per cent broken rice SAMPLED PRODUCT Brand A SAMPLED PRODUCT Brand B 4.7 Within each elementary aggregate, one or more products are selected to represent the price movements of all the goods and services in the elementary aggregate.
For example, the elementary aggregate consisting of rice sold in supermarkets in the northern region covers all types of rice, from which parboiled white rice and brown rice with over 50 per cent broken grains are selected as representative products.

Of course, more representative products might be selected in practice.

Finally, for each kind of representative product, a number of individual products can be selected for price collection, such as particular brands of parboiled rice.
Again, the number of sampled products selected may vary depending on the nature of the representative product.

4.8 The methods used to calculate the elementary price indices from the individual price observations col- lected within each elementary aggregate are explained in Chapter 9, and are not of immediate concern here.
Working upwards from the elementary price indices, all indices above the elementary aggregate level are des- cribed as higher-level indices that can be calculated from the elementary price indices using the elementary 60 expenditure aggregates as weights.

The aggregation structure is consistent, so that the weight at each level above the elementary aggregate is always equal to the sum of its components.
The price index at each higher level of aggregation can be calculated on the basis of the weights and price indices for its components, that is, the lower-level or elementary price indices.

The individual elementary price indices are not necessarily suﬃciently reliable to be published separately, but they remain the basic building blocks of all higher-level indices.

Above the level of the elementary aggregate, therefore, no new information is introduced into the calculation of the CPI.
Group, class and sub-class weights 4.9 The weights for the groups, classes and sub- classes are their shares in the total consumption expen- ditures of the reference population.

They are most often derived from household expenditure surveys (HESs).

These surveys are also described as household budget surveys (HBSs).
As these surveys are sample surveys subject to reporting and non-response errors as well as sampling errors, the estimated shares for certain sub- classes are often modiﬁed or revised on the basis of information from other supplementary or additional sources.

Regional weights 4.10 Within a given sub-class, the regional weight shows the consumption expenditure in the region in proportion to the expenditure in the whole country for that sub-class.
For example, if 60 per cent of the total expenditure on fresh fruit occurs in the North region and 40 per cent in the South region, then the regional weight for fresh fruit is 60 per cent for the North region and 40 per cent for the South region.

4.11 A region may also be a geographical area, a city or a group of cities, with a particular location or of a certain size.
The rationale for introducing regional weights is to create more homogeneous entities which are likely to experience similar price movements and have similar consumption patterns.

For example, there may be quite large diﬀerences in consumption patterns and price developments between urban and rural areas.

It may be necessary to distinguish diﬀerent regions in federal countries because CPIs for the provinces or local states may be required for administrative or political purposes.
In addition, in federal countries indirect taxes and hence price development may diﬀer between the provinces.

4.12 Regional weights may typically be obtained from the HES or they may be estimated from retail sales data or population data.

Regional weights may or may not be introduced into the CPI, depending on the size and structure of the country, data availability, resources and the purpose of the index.

EXPENDITURE WEIGHTS AND THEIR SOURCES aggregate weights speciﬁc to a given region and outlet type.
One advantage from applying outlet weights is that it may allow prices to be collected centrally from super- markets or other types of chain outlets.

Elementary aggregate weights 4.14 The elementary aggregate weights are the stratum weights according to expenditure class or sub- class, region and type of outlet.

For example, expendi- tures within the sub-class ‘‘fresh fruit’’ may be divided into four regions, each having its own regional weight, as in Table 4.1.
Assume further that it is known or estimated that 60 per cent is sold in supermarkets and 40 per cent in independent outlets, and that this same breakdown holds for all regions.

Let the weight of fresh fruit in the CPI for the whole country be, say, 5 per cent.

If no breakdown by region or outlet is carried out, then the sub-class as a whole becomes the elementary aggregate carrying a weight of 5 per cent in the overall index.
4.15 If weights are available by region but not by type of outlet, the 5 per cent is distributed over the four regions to obtain four separate elementary aggregates, the elementary one for each region.

For example, aggregate for the North region will have a weight of 0.20 0.05=1.0 per cent in the overall CPI for the whole country.
If a further division can be made according to type of outlet as well as region, then each region com- prises two elementary aggregates: one for supermarkets and one for independent outlets.

The weight for, say, the elementary aggregate for fresh fruit in the North region sold in supermarkets is then 0.12 0.05=0.6 per cent in the overall CPI for the whole country.
Data sources 4.16 The decision about what source or sources to use and how they should be used depends on an analysis of their respective advantages and disadvantages, and on the main purpose of the index.

In most countries, the two main sources for calculation of the weights are the HES and the national accounts estimates for households’ ﬁnal consumption expenditures.
Additional information may, however, be obtained from production and trade statistics, or from government departments, producers, marketing bodies and individual enterprises.

Such addi- tional information is particularly useful for estimating weights at the most detailed level.
Although several of the sources may have been used to prepare the national Table 4.1 Example of weights by region and outlet type for the sub-class ‘‘fresh fruit’’ Regional weights Type of outlet Supermarkets (60 per cent) Independent (40 per cent) Outlet or outlet-type weights 4.13 Prices are collected from a variety of outlets and outlet types.
Information about the sale or market share of the outlets may be used to form elementary North South West East Total 20 40 30 10 100 12 24 18 6 60 8 16 12 4 40 61 CONSUMER PRICE INDEX MANUAL: THEORY AND PRACTICE accounts estimates, they may be able to provide further details that were not used by the compilers of the national accounts.
Household expenditure surveys 4.17 As the HES may have been designed to serve more than one purpose, it is desirable to ensure that the survey design also meets the requirements for the CPI.

The main requirements are that the survey should be representative of all private households in the country, and not exclude any particular group, and should include all types of consumption expenditures by households.
4.18 The HES may include payments that are out- side the scope of the CPI: for example, payments of income taxes, life insurance premiums, remittances, gifts and other transfers, investments, savings and debt repayments.

These should be excluded from the total used to calculate the expenditure shares that serve as the basis for estimating the CPI weights.
There may also be a diﬀerence in the population coverage between the intended scope for the CPI and the actual scope of the HES, but the eﬀects on the CPI of any consequent bias in the resulting weight estimates are likely to be very minor if the HES is designed to provide results for the whole population and not just a particular population group.

4.19 National food surveys are special surveys in which the primary emphasis is on collecting information on family expenditures for food products.
These surveys provide a very detailed breakdown of food expenditures that can be used to derive the weights for elementary aggregates for food below the level of a COICOP class.

4.20 The HES may provide the basis for estimating speciﬁc weights for regions with diﬀerent consumption patterns.

These weights should be applied to the respec- tive elementary price indices to calculate indices for the regions concerned.
4.21 In general, HES data for certain types of expenditures may not be suﬃciently reliable and need to be checked against data from other sources.

Certain types of expenditures may not even be covered by an HES so that they have to be estimated using other sources.

The reliability of the CPI weights will obviously depend to a large extent on the reliability of the household expenditure data.
As the HES is a sample survey, the estimates are bound to be subject to sam- pling errors, which may be relatively large for small or infrequent expenditures.

The quality of the estimates will also suﬀer from non-response and from the under- reporting of some types of consumption.

Under- reporting is probably the most serious and common problem aﬀecting HESs.

Some expenditures are not reported because the purchases are small or exceptional, and therefore easy to forget.
Although large, estimates of expenditures on durable goods may also be proble- matic, since they are only purchased very infrequently.

Some expenditures are not reported because the pro- ducts have a social stigma or are illegal (e.g., drugs, alcohol and tobacco).

When no adjustments are made for such under-reporting, the consequence is an under- estimation of the weights for these items and an over- estimation of the weights for the correctly reported 62 items.
For these reasons, to the extent possible, results from the HES should be compared and/or combined with the statistics from other sources when constructing CPI weights, especially when the HES sample is small.

4.22 For the purposes of the CPI, it is desirable for the HES to be conducted annually.

This will allow coun- tries to revise and update their expenditure weights each year.
One advantage of annual updating of weights is that the diﬀerences between the results obtained from the use of diﬀerent index number formulae tend to be reduced.

Any bias which may follow from using a Lowe index that uses a ﬁxed basket of goods and services will not have time to accumulate to a signiﬁcant magnitude, as explained in Chapters 1, 9 and 15.

4.23 Some countries conduct continuous HESs with gradually rotating samples.
A programme of annual surveys with samples large enough to provide the type of estimates required for CPI weights can, however, be very costly.

For this reason, some countries conduct large- scale surveys at ten-year or ﬁve-year intervals, perhaps supplemented with a smaller annual sample.

Other countries distribute a large sample over several years.

The average of the results over several successive years of smaller-scale surveys may provide a set of satisfactory annual estimates.
The weights derived in this way as the average rates of expenditure over periods of two or three years will also smooth any erratic consumer behaviour over a short period, for example as a result of events such as droughts or ﬂoods, civil strife, oil shocks, or exceptionally mild or cold winters.

4.24 It should be noted that in some countries it may be possible to experiment with new methods of record- ing expenditures in an HES by using scanner data gen- erated by electronic points of sale.
For example, by collecting the printed bar code receipts for cash which customers receive, the Icelandic HES could obtain, at virtually no cost to the surveyed households, precise information about types and brands of goods purchased in diﬀerent outlets.

National accounts 4.25 There may be diﬀerences in the scope and deﬁ- nition of consumption between the national accounts and the CPI, and also a diﬀerence in the reference population of households between the national accounts and the HES.
4.26 First, in national accounts, the household sec- tor consists of all resident households, including people living in institutional households.

HESs, however, do not usually cover persons living permanently in institu- tional households, such as retirement homes or religious institutions.

If the CPI is meant to cover all resident households, therefore, national accounts estimates may be used to adjust the HES data.
4.27 Second, as already explained in Chapter 3, it is possible to have two alternative concepts of total ﬁnal consumption, domestic and national.

The domestic con- cept refers to consumption on the economic territory, including the consumption of visiting foreign households but excluding the consumption of resident households when abroad.
The national concept used in national accounts refers to the consumption of all the residents of the country, whether at home or abroad, the consump- tion of non-residents being excluded.

The HES usually covers only resident households, and may or may not cover their expenditures abroad, depending on the instructions given to the respondents.

4.28 National accounts data may be used to improve HES weights for products that are under-reported in the HES.
Note that national accounts ﬁgures for house- holds’ ﬁnal consumption are usually based on statistics from the HES and from several other sources.

This means that national accounts estimates are likely to be useful for estimating weights for consumption categories that tend to be wrongly reported in the HES, and where results from the HES suﬀer from a signiﬁcant and dis- torting partial or total non-response rate.
Retail sales data 4.29 Statistics on retail sales by region and type of outlet may be available for broad groups of items.

One disadvantage is that some of the sales may be to groups outside the reference population, perhaps to the business sector or to the government.

The corresponding pur- chases do not form part of household private consump- tion.

Some sales may also be to non-residents, who may or may not be part of the reference population.
Furthermore, for regional sales data, it needs to be kept in mind that sales may include sales to people living in other regions.

Point-of-purchase surveys 4.30 Point-of-purchase surveys may provide statis- tics that can be used to estimate weights for price data, as they permit the analysis of shopping patterns for various segments of the population.
Households are asked, for each item purchased, about the amounts spent in each outlet where purchases have been made, and the name and addresses of these outlets.

On this basis, a list of outlets can be established, with the total sales for all the diﬀerent items to the sample of households.

A sample of outlets is then drawn from this list, with probability proportional to the sales.
Given that household surveys are expensive and that there is an overlap between the HES and point-of-purchase survey, it is possible to combine the two data-collecting activities in an inte- grated survey that elicits expenditure and outlet data at detailed levels, along with the demographic information about the households needed for sub-group indices.

4.31 A simpler version of this survey may be con- ducted to obtain weights for groups of products by outlet type.
In this case, a purposive sample of each outlet type should be selected.

As an alternative, in the absence of this type of survey, national retail sales sta- tistics by outlet type from a survey of outlets could be used to estimate a breakdown of sales by outlet type.

EXPENDITURE WEIGHTS AND THEIR SOURCES tronic data records that are stored as scanner data in the databases of sellers.

Such scanner data sets include the quantities sold and the corresponding value aggregates.
(The cash register receipts usually give the following information: name of the outlet, date and time of pur- chase, description of items bought, quantity, price and value, form of payment, and VAT amount where rele- vant.)

A comparison of the results from the HES with the corresponding scanner data from the biggest super- market chains indicates that the use of scanner data can add to the reliability of the weights (Guðnason, 1999).
This strengthens the case for using such data to revise CPI weights more often than otherwise would be pos- sible, and probably at lower cost.

The limitations of this source of information should, however, be borne in mind.

The ﬁrst one is that scanned data cannot be con- nected to a speciﬁc type of household, whereas the data from the HES can.
Another important diﬀerence between HES data and scanner data from sellers is that the HES data cover goods bought from outlets that are not using this technology, as well as goods and services that do not carry scanner codes, regardless of where they are sold.

Although the use of electronic data records is increasing every year, signiﬁcant components of the retail trade market are not using scanner data even in countries that are electronically most advanced.
Population censuses 4.33 Population censuses provide statistics on the geographical distribution of the population and house- holds, as well as on the regional diﬀerences in household size and composition.

Combined with estimates of regio- nal levels of household expenditure, these statistics can be used to estimate regional expenditure weights, especially when such estimates are not available from an HES with a satisfactory degree of precision.
In the absence of any expenditure statistics, population statistics might be used as the basis for regional weights.

Such estimates for the weights usually have to assume that expenditures per capita or per household are the same in all regions, and have to ignore the fact that there are usually large dif- ferences between the urban and rural populations in the level and pattern of items that they consume.
Deriving the weights in practice 4.34 Once the reference population and the coverage of goods and services have been decided, the weights need to be derived.

In principle, this is a relatively simple matter, as the weights are calculated as the proportions of the total consumption expenditure of all goods and services included in the index basket for the reference population during the reference period.
In practice, however, the calculation of weights is not so straight- forward and involves a series of steps.

Scanner data 4.32 In the last few years some countries have star- ted to use statistics obtained from cash register data to derive CPI weights.

These statistics are based on elec- Payments that are not consumption expenditures 4.35 Only consumption expenditures are relevant for the construction of CPI weights.
As explained in 63 CONSUMER PRICE INDEX MANUAL: THEORY AND PRACTICE Chapter 3, outlays such as payments of social security contributions or income taxes, or repayments of debts, are irrelevant and should be ignored because they are not consumption expenditures.

weights if they are within the scope of the index.

For diﬃcult-to-price products, the options available are the same as those used for unimportant expenditures.
Unimportant expenditures 4.36 Each elementary aggregate consists of a fairly homogeneous group of products from which one or more representative products are selected for pricing.

Some products may have a weight which for all practical purposes is negligible and for which prices are unlikely to be collected in practice.
The HES, which in most cases is the main source for deriving the detailed weights, usually includes observations on a much larger variety of goods and services than it is practical to collect prices for.

The prices of very minor products may not be worth collecting if they contribute almost nothing to the CPI.

4.37 Even though it may be decided not to collect prices for a certain product, it remains within the scope of the CPI.
Some price change has to be explicitly or implicitly assumed, or imputed, and weighted by ex- penditures.

There are two options: The product and the expenditures on it remain within the elementary aggregate, even though no prices are collected for it.

The elementary price index for the aggregate as a whole is estimated entirely by the prices of the representative products for which prices are collected.
This is equivalent to assuming that the price of the product changes at the same rate as the average for the prices of the representative products.

 The alternative is to reduce the weight for the ele- mentary expenditure aggregate by excluding the expenditures on the product.

This is equivalent to assuming that the price of the excluded product would have moved in the same way as the overall CPI for all the products actually included in the index.
4.38 In principle, the CPI should cover all types of products and expenditures within its scope, even if prices are not collected for some products.

It might be decided, for example, to exclude from the index calculations groups with weights lower than, say, 0.1 per cent for food groups and 0.2 per cent for non-food groups.
A lower minimum threshold for the food items might be set because the prices for these products tend to display greater variability and because prices for food products are normally less expensive to collect.

If an expenditure group is excluded, its weight may be redistributed to another expenditure group that is similar in terms of content and price development.

Alternatively, the expen- ditures may be completely excluded from the calculation of the weights.
Products that are difﬁcult to price 4.39 Among the consumption expenditures, there are likely to be a few products for which the prices, or price changes, cannot be directly or satisfactorily mea- sured, for example, illicit drugs or payments for catering and other service charges for private receptions and parties.
Even if reliable prices cannot be obtained, these products should be included in the calculation of the 64 4.40 Use and combination of different sources In most countries, the main source for deriving the weights is the HES.

As noted above, however, the results from the HES need to be carefully examined and adjusted to take account of under- or over-reporting of certain types of products.
The usual strategy is to use supplementary information from other relevant sources to adjust the HES results in order to derive the weights.

4.41 In countries where national accounts data provide reliable estimates of household expenditures, these data can be used to derive the weights at an aggregate level.

Detailed HES data can then be used to break down or adjust these weights.
In this way, it is possible to reconcile the detailed data from the HES with the aggregate national accounts data to calculate the weights.

Weights for the main consumption groups can be obtained from the national accounts down to a certain level of disaggregation, say, 70 consumption groups or classes.

Each of these weights can then be further distributed by applying the detailed HES expen- diture groups to the national accounts consumption groups or classes.
The combination of national accounts and HES data ensures consistency between the CPI and the national accounts data on consumption expenditure of households at the level of the main consumption groups.
Adjusting the weights derived from household expenditure surveys 4.42 As, in most cases, the information from a house- hold expenditure survey is only available with a lag – often around 18 months or more – the new weights will lag behind the new price reference period for the index, which is the period when the new weights are introduced.
4.43 Adjustments might need to be made to the estimates based on the HES results to take into account any signiﬁcant changes in expenditure patterns in the period between the time that the survey was carried out and the time that the new weights were introduced.

Adjustments will typically be made for products which are signiﬁcantly losing or gaining in importance during this period.
It is also possible that expenditure on some products may not be available from the HES because the products appeared on the market after the survey had been completed.

One example is mobile telephones and the corresponding charges, which emerged as sig- niﬁcant new forms of expenditure in the late 1990s in many countries.

Necessary adjustments then need to be made to the survey data to take into account the changes that have occurred.
The expenditures on these new products should be estimated on the basis of information available from other sources (e.g., import and retail trade statistics), taking into account the need to exclude expenditures by enterprises and for business purposes.

Weight reference period 4.44 The weight reference period is the time period to which the estimated weights relate.

The choice of the period covered by the expenditure statistics used to derive the weights is crucial.
Generally speaking, the period chosen as the base should be long enough to cover a seasonal cycle.

Further, if the index is not annually chained, the year chosen should have economic conditions that can be considered to be reasonably normal or stable.

To achieve this, it may be necessary to adjust some of the values to normalize them and over- come any irregularities in the data for the particular period that constitutes the source of the information.
The weight reference period should not be too distant from the price reference period.

The weight reference period is typically a single calendar year.

A month or quarter is too short a period to use as a weight base period, since any one month or quarter is likely to be aﬀected by accidental or seasonal inﬂuences.

In some instances, data for a single year may not be adequate either because of unusual economic conditions or because the sample is not large enough.
An average of several years of expenditure data may then be used to calculate the weights.

Countries in which this method is applied include the United States and the United Kingdom.

In the United States, the expenditure infor- mation from the Consumer Expenditure Survey over a three-year period is used.
In the United Kingdom, an average of three years of Expenditure and Food Survey data is used for regional weights, for stratiﬁcation and for a limited number of groups of products where prices tend to be particularly volatile.

4.45 During periods of high inﬂation, multiple year weights may be calculated by averaging value shares rather than averaging actual value levels.

Averaging the value levels will give too much weight to the data for the most recent year.
Another option is to update the values for each year to a common period and then to compute a simple arithmetic average of adjusted yearly data.

4.46 As the weight reference period usually precedes the price reference period, the expenditure weights may be price updated to take account of the relative price changes from the weight reference period to the price reference period.

Price updating of weights is discussed in more detail in Chapter 9, paragraphs 9.95 to 9.104.
Need for revising the weights 4.47 Most countries calculate their CPI as the change in the value of a speciﬁed basket of goods and services.

An index of this general kind is described in this manual as a Lowe index.

Its properties and behaviour are explained in Chapters 1, 9 and 15.

Although CPIs are often described as Laspeyres indices, they are usually not Laspeyres indices in practice.
A Laspeyres index is deﬁned as an index in which the basket of goods and services is that of the price reference period, but a typical CPI basket uses the basket of some weight reference period that precedes the price reference period, as just explained.
As many countries continue to use the same ﬁxed basket of goods and services over a period of years, the question arises of how often the basket should be EXPENDITURE WEIGHTS AND THEIR SOURCES revised in order to ensure that it does not become out of date and irrelevant.

4.48 In the short run, consumers will change con- sumption patterns in response to shifts in relative prices, mostly between products included in the same class or sub-class.
Over longer time periods, consumption patterns are also inﬂuenced by factors other than price changes.

Most importantly, changes in the level and distribution of household income will cause a shift in demand for goods and services towards goods and services with higher income elasticities.

Demographic factors such as ageing of the population, and technological changes, such as the increase in the use of computers, are examples of other factors that aﬀect spending behaviour in the longer run.
Furthermore, new products will be introduced and exist- ing ones may be modiﬁed or become obsolete.

A ﬁxed basket will be unresponsive to all these changes.

4.49 As a result of both relative price changes and long-term eﬀects, the weights may become out of date and less representative of current consumption patterns.

As shown in Chapter 15, the bias in a Lowe index is likely to increase with the age of the weights.
At some point, it therefore becomes desirable to use the weights of a more recent period to ensure that the index is weighting appro- priately the price changes currently faced by consumers.

Frequency of updating the weights 4.50 The 1987 ICLS resolution concerning consumer price indices recommended that the weights should be updated periodically, and at least once every ten years, to guarantee the representativity of the index.
However, the 2003 ICLS resolution proposes more frequent updates of the weights, such as once every ﬁve years, to ensure their relevance.

Countries which are experiencing signiﬁcant economic changes and thus more rapid changes in consumption patterns should update their weights even more frequently, say annually.

4.51 The need to revise the weights generally increases as the length of time from the weight reference period increases.
The decision when to update the weights depends, for the most part, on the diﬀerences observed between the current weighting structure and that for the weight reference year.

Changes in the rela- tive importance of each item can be observed through expenditure survey results.

If these statistics are avail- able only at irregular intervals, the frequency of weight revision may necessarily be linked to the availability of results from the HES.
4.52 The introduction of new weights each year might possibly cause an upward drift in the index if there are big ﬂuctuations in consumption caused by factors such as an economic blockade, or extremely favourable or unfavourable weather conditions.

In general, the pro- ﬁle of the index time series can be sensitive to the selec- tion of the weight reference period.
It might be best to use a ‘‘normal’’ consumption period, if possible, as the basis for weighting information and to avoid periods in which there are special factors at work of a temporary nature.

All available information concerning the nature of con- sumption in a weight reference period should be taken into consideration.
65 CONSUMER PRICE INDEX MANUAL: THEORY AND PRACTICE 4.53 When the weights are to be ﬁxed for several years, the objective should be to adopt weights that are not likely to change much in the future rather than precisely reﬂect the activity of a particular period that may be abnormal in some way.

4.54 Each year it is desirable to carry out a review of the weights in order to ensure that they are suﬃciently reliable and representative.
The review, which may be conﬁned to weights at the level of sub-indices and their major components, should examine whether or not there are indications that important changes may have taken place in the consumption pattern since the weighting reference period.

4.55 Whenever the weighting pattern has been updated, the new index using updated weights should be calculated for an overlapping period with the old one so that the two can be linked.
Classiﬁcation 4.56 In deriving the weights, the detailed expenditure items identiﬁed in the HES must be mapped to the CPI expenditure classes.

If HES classes do not match CPI expenditure classes, the HES results must be transformed to match the CPI categories.

This can be done by ag- gregating or disaggregating the relevant HES headings over the relevant CPI expenditure classes.
Such trans- formation is achieved much more easily and more reli- ably if the coding list for expenditure items in the HES is coordinated with the corresponding list of items used for collecting price observations for the CPI.

4.57 For the purposes of international comparison, the classiﬁcation scheme of goods and services should, to the extent practical, be in line with the United Nations Classiﬁcation of Individual Consumption according to Purpose (COICOP) (see Annex 2).
To facilitate estimation and application of weights, it is also desir- able that the classiﬁcation used be consistent with the classiﬁcations used for HESs and other statistics (for example, retail sales statistics).
In the interests of maintaining both coordination of the statistical system and international comparability, the HES should also use a classiﬁcation for types of expenditure that will be consistent with COICOP, and it should also be possible to establish a mapping between products in the retail sales price collections and COICOP.

Another impor- tant objective is that the aggregation structure employed by the classiﬁcation system should meet the major needs of users.
4.58 Using COICOP as an example, the classiﬁca- tions have the following hierarchical structure: – groups: there are 47 of these in COICOP; – classes: sub-divisions of the groups; – sub-classes: categories the lowest-level that are weighted and usually the most detailed level of the structure for which index series are published – these are the expenditure components and weights that remain ﬁxed when using a ﬁxed weight index; – individual products: the lowest level of the CPI basket, that is, the individual goods and services for which prices are actually collected – this is the level at which 66 the composition of the CPI basket can be adjusted between two major revisions of the weighting struc- ture to reﬂect changes in product supply and con- sumer behaviour.
4.59 Upper-level indices are formed by weighting together lower-level indices through progressive levels of aggregation, as deﬁned by the classiﬁcation structure.

Weights are ﬁxed for a period (say one, three or ﬁve years) between index reweighting.

4.60 The selection of the level in the index hierarchy at which the structure and weights are ﬁxed for a period is particularly important.
The main advantage of setting the level relatively high is that the actual samples of products and their prices below this level can be adjusted and updated as needed.

New products can be introduced into the samples, and the weights at the lower level re- established on the basis of more recent information.

There is thus a greater opportunity to keep the index representative, through an ongoing review of the sample of representative products.
4.61 If the level is set relatively low in the index structure, there is less freedom to maintain the repre- sentativeness of the index on an ongoing basis, and there will be a greater dependence on the periodic index review and reweighting process.

In such circum- stances, the arguments for frequent reweighting become stronger.
Items requiring special treatment 4.62 Some products, such as seasonal products, insurance, second-hand goods, expenditures abroad, etc., may need special treatment when constructing their weights.

Reference may be made to Chapters 3, 10 and 22 for further details.

4.63 Seasonal products.


Various approaches may be used to deal with seasonal products, for example: a ﬁxed weights approach, which assigns the same weight for the seasonal product in all months, using an imputed price in the out-of-season months.

Sea- sonal products are treated in the same way as other consumption products; a variable weights approach, in which a changing (or moving) weight is attached to the product in various months.
In this method, the weights of the seasonal products change monthly according to changes in the quantities consumed during the diﬀerent months of the weight reference period.

The principle of a ﬁxed basket – i.e.

ﬁxed weights – should, however, be maintained at least at some level of aggregation.

4.64 The advantage of applying the ﬁxed weights method is mainly that it is consistent with the methods used for other consumption goods and services, and with the ﬁxed basket index formula.
In contrast to the moving weights method, the ﬁxed weights method re- ﬂects monthly changes in prices only, and not in quan- tities.

Another disadvantage of the moving weights method is that the weights are based on the monthly seasonal ﬂuctuations in the weight reference period, whereas the monthly ﬂuctuations in consumption may diﬀer every year.
4.65 The ﬁxed weights method may also have dis- advantages, a major one being that during the months that fresh fruits or vegetables disappear, prices and indices have to be estimated or imputed for these items (or, as is done in some countries, prices and indices the period of dis- have to be frozen throughout appearance).

These imputations need not be made when applying the moving weights method.
In addition, the average ﬁxed weight determined for all months of the year does not actually reﬂect the monthly consumption.

Therefore, if there is a negative correlation between prices and quantities, there may be an upward bias in the index.

4.66 The choice of measuring seasonal goods according to the ﬁxed weights method or the moving weights method should be based on whether the focus is on month-to-month changes or on the long-term index changes.
The use of an annual basket and the use of annual expenditure shares are appropriate where the main interest is in the longer-run trend of price changes.

On the other hand, if the focus is on month-to-month changes, then the annual weights attached to each month-to-month price relative can be unrepresentative of actual transactions that are taking place in the two consecutive months under consideration.
In the latter case, monthly price changes for items that are out of season can be greatly magniﬁed by the use of annual weights.1 To satisfy the needs of diﬀerent users, it may be appropriate to construct two indexes: one for the short-term measurement of price changes (with variable monthly weights) and another longer-term index (with ﬁxed annual weights).

The issue of seasonal items is dealt with in detail in Chapter 22.

4.67 Insurance.
4.67 Insurance.

As explained in the section on insurance in Chapter 3, the weights for non-life insur- ance could be based on either the gross premiums paid or on the implicit service charges.
The implicit service charges for administering the insurance and providing the insurance services are estimated by the gross pre- miums plus the income from investment of the insurance reserves less the amounts payable to policy holders in settlement of claims.2 The net premiums are deﬁned as the gross premiums less the service charges: in other words, the net premiums equal the claims.

The net premiums and claims can be regarded as transfers, or redistributions, between policy-holding households.
In general, it seems preferable to base the weights for non- life insurance on the service charges.

These are the estimated amounts paid by households for the services provided by insurance ﬁrms.

However, a case can also be made for basing the weights on the gross premiums.

This is a diﬃcult area in which there is not yet a con- sensus.

1For example, the impact of change in tomato prices at the beginning of the season would be overstated in the general index.
Similarly, its impact in the peak months would be understated.

2In the national accounts, the gross premiums plus the investment income less the estimated service charges are described as ‘‘net pre- miums’’.

By deﬁnition, ‘‘net premiums’’ equal claims payable, both ﬂows being treated as transfers, or redistributions, between policy- holding households.

The ‘‘net premiums’’ are not regarded as expen- ditures.

EXPENDITURE WEIGHTS AND THEIR SOURCES 4.68 Second-hand goods, including used cars.
As already explained in paragraphs 3.127 to 3.129 of Chap- ter 3, the prices of used or second-hand durable goods purchased by households are included in the CPI in the same way as the prices of new goods.

However, house- holds also sell used durables, such as cars.

If the price of a second-hand good rises, a purchasing household is worse oﬀ, but a selling household is better oﬀ.
From a weighting perspective, sales constitute negative expenditures, which implies that price changes for used goods sold by house- holds implicitly carry a negative weight in the CPI.

In eﬀect, purchases and sales of second-hand goods between households, whether directly or indirectly through a dealer, cancel out (except for the dealers’ margins, see Chapter 3) and carry no weight in the CPI.

However, households also buy from, and sell to, other sectors.
For the reference population as a whole, namely the entire set of households covered by the CPI, the weight to be attached to a particular kind of second-hand good is given by households’ total expenditures on it less the value of the households’ receipts from sales to/from out- side the household sector.

There is no reason why these should cancel out on aggregate.

For example, many of the second-hand cars purchased by households may be imported from abroad.
The diﬀerence between total expenditures and total sales is usually described as households’ net expenditures.

This is the weight to be attached the second-hand good in question.

4.69 Except in the case of used cars, however, it is practically impossible to estimate the net expenditure because most HESs do not collect the data that would allow for a comparison between expenditures and receipts from sales of individual kinds of second-hand goods.
Usually, only the total amount received from the sale of second-hand goods is collected.

This information does, however, give an idea of the volume and signiﬁcance of these transactions in the national economy.

In countries where this volume is small, second-hand goods (except used cars) may be ignored when calculating the weights of the index.

4.70 Because the amounts spent on purchasing second- hand cars are usually large, they should be included in the CPI basket if the data are available.
In the absence of reliable data, however, their weight can be added to the weight of new cars.

4.71 Most countries include expenditure on second- hand goods in the estimation of CPI weights, but second- hand goods are not covered in the price collection (because of the diﬃculty of pricing the same good each month or, where the goods are diﬀerent, making an appropriate quality adjustment).

It is therefore assumed that the prices of new and second-hand goods move in the same way.
4.72 In countries where second-hand purchases are important and their prices are believed to change at diﬀerent rates from those of new goods, separate weights are needed for them.

The information could be obtained, at least for some major durables, from HESs, if the surveys ask about expenditure on second-hand and new goods.

4.73 Expenditure abroad and expenditures by non- residents.
If the objective is to construct an index which 67 CONSUMER PRICE INDEX MANUAL: THEORY AND PRACTICE is representative of price movements within a given country or area, the weighting system must reﬂect pur- chases by both resident and non-resident households.

In practice, the proportion of total purchases that are made by visitors from abroad or other areas may be diﬃcult to estimate, except for certain types of purchases in geo- graphical areas where foreign tourism is the dominant economic activity.
Sources other than HESs must be used in order to ensure that the weights include the expenditures made by foreign tourists and reﬂect all purchases of consumer goods and services made by resident or non-resident households within the country.

These sources may be national accounts or commercial sales statistics.

4.74 Where the main purpose of the index is to measure price changes experienced by the resident population, the weights should include their expendi- tures abroad.
This would require collection, through the HES, of data on expenditures made outside the country (for example, expenditures on hotels and meals during holidays, durables, health and education).
Possible ways of constructing the index to cover expenditure abroad would be: – price collection outside the country of residence; – the use of appropriate sub-indices provided by the statisticians in other countries for the kinds of pro- ducts purchased there by residents; – establishing a panel of residents who would report prices paid for their purchases abroad.
4.75 Given the limitations of HESs to provide reli- able data on expenditures abroad, and the practical diﬃculties of constructing an index for expenditure abroad, the weights may have to be based on expenditure surveys without adjusting for the place of acquisition, and prices may be collected only for the goods and ser- vices acquired in the economic territory of the country.
Such an approach assumes that the price changes of the goods and services acquired abroad are the same as those for the same goods and services acquired at home.

Errors in weighting 4.76 If all prices moved in the same way, weights would not matter.

On the other hand, the greater the variation in price behaviour between products, the greater the role of weights in measuring aggregate price change.

4.77 Small changes in the weights usually have very little eﬀect upon the overall CPI.
An error in the weight for a given sub-index only matters to the extent that the change in the sub-index diﬀers from the average change in the overall CPI.

In general, the higher a sub- index’s weight, the lower is the tolerable percentage error in that weight.

It follows that the tolerable error in the weights declines as the rate of relative price change for the relevant items increases.
Finally, it is also clear that while errors in weighting may not have a large inﬂuence on the overall index, the sub group-level errors could be signiﬁcant.

Australian experience shows that even items with relatively large weights can tolerate errors of 20–30 per cent in the weights (Australian Bureau of Statistics, 2000).

According to Eurostat’s studies, CPIs are fairly insensitive to changes in weights.
Eurostat has, however, suggested developing quality control procedures for monitoring the weights of items for which changes in prices have diverged from the movement of the overall index (Eurostat, 2001).

The question of the eﬀects of weighting errors on the sub- index and the overall index is discussed in Rameshwar (1998).

68 SAMPLING Introduction 5.1 The procedure used for price collection by a national statistical oﬃce in the production of a consumer price index (CPI) is a sample survey.
In fact, in many countries, it might be better viewed as composed of many diﬀerent surveys, each covering diﬀerent subsets of the products covered by the index.

We will therefore begin by outlining some of the general concepts of survey sampling which need to be kept in mind when looking at a particular survey such as price collection for a CPI.

5.2 There is a target quantity, for example a CPI, which is deﬁned with respect to: – a universe consisting of a ﬁnite population of units (e.g.
products); – one or more variables that are deﬁned for each unit in the universe (e.g.

price and quantity); – a formula which combines the values of one or more of these variables for all units in the universe into a single value called a parameter (e.g.

the Laspeyres index).

The interest is in the value of this parameter.

5.3 The universe usually has three dimensions.

There is a product dimension, consisting of all purchased products and varieties of products.
There is a geo- graphical and outlet dimension consisting of all places and channels where a product is sold.

Finally, there is a time dimension consisting of all sub-periods within an index period.

The time dimension will be given less attention since price variation is usually smaller over a short time span and since temporal aspects may be dealt with in product and outlet speciﬁcations.
5.4 In this chapter, the ﬁrst two dimensions will be regarded as being static over the time periods considered in the index.

In other words, it will be assumed that the same products and outlets are in the universe in both periods, or that replacements between old and new prod- ucts or outlets are one to one and without problems.

For the complications arising from dynamic changes in the universe, please refer to Chapter 8, where replacement, resampling and quality adjustment are discussed.
5.5 Why take only a sample of units?

Apart from the near physical impossibility and prohibitive cost of trying to cover all products in all outlets, the data are likely to be of better quality if there are fewer units to deal with because of the use of more specialized and better trained data collectors.

Also, the time required to complete the exercise is shorter.
5.6 In probability sampling, the units are selected in such a way that each unit (an outlet or a product) has a known non-zero probability of selection.

For example, outlets are selected at random from a business register in 5 which each outlet has an equal chance of being selected.

Traditionally, however, non-probability sampling meth- ods have mainly been used in the compilation of a CPI for choosing outlets or products.

The representative item method is particularly popular for selecting items.
Other methods used are cut-oﬀ sampling and quota sampling (see below).

There are also instances of a mix- ture of the two methods of sampling; for example, outlets are selected using probability sampling techniques, whilst products are selected using the representative item method.

5.7 Having decided to sample, there are two issues to be considered: how to select the sample; and how to use the sample values to estimate the parameter.
The former reﬂects the choice of sampling design, and the latter constitutes the estimation procedure.

We ﬁrst take a look at sampling design.

Probability sampling techniques 5.8 This section presents some general concepts and techniques of survey sampling that have important applications for price indices.

This brief presentation covers those concepts of survey sampling that are of immediate interest in price index applications.
For a full treatment of the subject, please refer to one of the many textbooks available, for example Sa¨ rndal, Swensson and Wretman (1992) or Cochran (1977).

5.9 Survey sampling theory views the universe as composed of a ﬁnite number (N) of observational units denoted j=1, .

.

.

, N. Sampling then amounts to selecting n units out of N by attaching an inclusion probability, pj, to each unit.

For price indices there are two sampling designs that are of particular interest.
5.10 In simple random sampling and systematic sam- pling each unit is sampled with equal probability and we have pj = n/N.

In simple random sampling, all units are selected using a random mechanism.

In systematic sam- pling, the sampling units are selected at equal distances from each other in the frame, with random selection of only the ﬁrst unit.

These techniques are usually recom- mended in situations where the units are relatively homogeneous.
5.11 In probability proportional to size (pps) sampling the inclusion probability is proportional to some aux- iliary variable xj and we have pj=nxj=PN j=1xj.

Units for which initially this quantity is larger than one are selected with certainty, whereafter the inclusion probabilities are calculated for the remainder of the universe.

5.12 The universe may be divided into strata, denoted h=1, .

.

.
.

.

, H. In each stratum, there are then Nh 69 CONSUMER PRICE INDEX MANUAL: THEORY AND PRACTICE units and we have PH h=1Nh=N.

The purpose of strati- ﬁcation is usually to group units together that are either homogeneous in some sense or satisfy some adminis- trative convenience such as being physically close to- gether.

Each stratum is a mini-universe with sampling taking place independently in each one.

In a CPI, the practice is to use elementary aggregates as strata.
In the remainder of this chapter, we look at sampling in a single stratum, corresponding to an elementary aggre- gate, and drop the subscript h. Implementing probability sampling in consumer price indices 5.13 A sampling frame is a list of all (or most) of the N units in the universe.

A frame may have overcoverage to the extent that it includes units that are not in the universe or includes duplicates of units.
It may have undercoverage to the extent that some units in the uni- verse are missing from the frame.

5.14 Sampling frames for the outlet dimension could be: Business registers.

These should include locations of retail trade businesses with addresses and be updated regularly.
If a size measure (turnover or number of employees) is included in the register, it is a useful device for performing probability proportional to size (pps) sampling and this size measure would then be included in the universe parameter also.

 Telephone directories (‘‘yellow pages’’).

These usually do not include size measures so simple random sam- pling or systematic sampling would then be necessary.
Sometimes informal knowledge of the importance of diﬀerent outlets could be used to stratify the universe into two or more categories and then draw a relatively larger sample from the more important strata.

 Records of local administrations, organizations of enterprises, and so on, could be used for local markets and suchlike, which are especially important in devel- oping countries.
5.15 Sampling frames for the product dimension could be: product lists provided by major wholesalers showing sales values for varieties in an earlier period.

Sales values provide an obvious size measure for weights and pps sampling; outlet-speciﬁc lists of products.

These lists could also be drawn up by the price collectors themselves by noting the products displayed on the shelf.

Shelf space could then be used as a size measure for pps sampling.
Sampling techniques based on probability proportional to size 5.16 Several techniques exist for drawing pps sam- ples.

They fall into two main categories according to whether the size of the sample is ﬁxed or random.
A ﬁxed, predetermined sample size is clearly desirable for CPIs since the sample size in each stratum is often small and a random size would entail the risk of an empty 70 Table 5.1 Systematic sample of 3 out of 10 outlets, based on probability proportional to size Outlet Number of employees = x Cumulative x Inclusion interval Included when starting point is 1–13 14–15 16–20 21–29 30 31–55 56–65 66–71 72–82 83–90 X X X sample.
We therefore present two techniques here that provide ﬁxed size pps samples.

5.17 Systematic pps sampling.

The procedure is best explained by an example.

In Table 5.1 we show how a sample of 3 outlets can be drawn from 10.

In this case we have the number of employees as our size measure.

We look at the list, where we have included the cumu- lative sizes and the inclusion intervals.

We take the total number of our size measure, which is 90 in this case, and divide it by the sample size, 3.
This gives us a sampling interval of 30.

We next choose a random number between 1 and 30 (random number functions are given in, for example, the Excel spreadsheet software).

Say that we get 25.

The sample will then consist of the outlets whose inclusion intervals cover the numbers 25, 25+30 and 25+2 30.

5.18 Systematic sampling is easy to execute.

If, however, the frame has some overcoverage, the sample size will not be the predetermined one.
Let us say that, at the ﬁrst visit to the outlets, we discover that outlet 6 does not sell the products in the product sample.

We would then be left with a sample of only two outlets.

We would either be content with that, or somehow seek a replacement for the invalid outlet, which is not deter- mined by the basic sampling procedure.

Moreover, the selected sample depends on the order in which the outlets or products are listed.
This might be important, especially if the listing order is correlated to the size measure.

5.19 Order pps sampling.

This is a relatively new method for drawing pps samples.

Rose´ n (1997a, 1997b) gives its theory.

In this case, a uniform random number Ui between 0 and 1 and a variable zi=nxi/P ixi, where xi is a size variable, are associated with each sampling unit and a ranking variable is constructed as a function of these two variables.
The units in the universe are then sorted in ascending order and the n units with the smallest values of the ranking variable are included in the sample.

Two important examples of such ranking variables Qi are: for sequential pps sampling: Qi=Ui/zi; for Pareto pps sampling: Qi=Ui (1zi)/zi (1Ui).

5.20 For the same universe as above and with Pareto pps as our example, we show in Table 5.2 how this works.
We have now ordered the universe in ascending order with respect to the ranking variable.

Our ﬁrst Table 5.2 Pareto sample of 3 out of 10 outlets, based on probability proportional to size Outlet 6 1 8 9 10 7 4 3 5 2 xi 25 13 6 11 8 10 9 5 1 2 Ui 0.755509 0.198082 0.915131 0.277131 0.834138 0.709046 0.46373 0.500162 0.067941 0.297524 Qi 0.036943 0.207721 0.310666 0.346024 0.380468 0.412599 0.580264 1.25 1.836435 2.926051 Sample X (X) X X sample turns out to consist of outlets 6, 1 and 8.
Say that we now discover, however, that it is inappropriate to include outlet 1.

We then turn to the fourth unit in order – outlet 9 – and include that one instead.

Thus, order pps sampling is easy to combine with a ﬁxed sample size and more ﬂexible than systematic sampling.

5.21 Neither of the two order sampling procedures is, however, exactly pps, because the obtained inclusion probabilities vary somewhat from the desired ones.
Rose´ n (1997b) shows, however, that for the purpose of estimating means and variances, they are approximately pps.

In the case of the price index, this still holds when there is sample substitution resulting from overcoverage.

Pareto pps is marginally better than sequential pps and should therefore be preferred.
5.22 Order pps sampling is at present used in many areas of the Swedish CPI, for example for sampling: outlets from the business register (the size measure is number of employees +1); products from databases provided by major retail chains (the size measure is historic sales); car models from the central car register (the size measure is number of cars registered in the reference period).
5.23 Further details on the application of these procedures are given in Statistics Sweden (2001).

Rose´ n (1997b) shows that Pareto pps and systematic pps are the two optimal pps sampling methods.

Pareto pps permits an objective assessment of estimation precision.

With regard to ﬁnal precision, however, Pareto pps is best in some situations whereas systematic pss is best in other situations.

The choice between them is therefore a matter of judgement and practicality in a particular sampling situation.
The great ﬂexibility of order pps sampling with regard to imperfections in the frame, an aspect of importance in CPI applications, leads us to make this procedure our ﬁrst recommendation among pps pro- cedures.

Sampling methods used by the US Bureau of Labor Statistics 5.24 The US Bureau of Labor Statistics (BLS) uses probability methods in all stages of sample selection.
In the last stage, individual items in outlets are selected in a process designed to approximate pps sampling with respect to the sales of each such item.
To this end, the SAMPLING BLS ﬁeld representatives are allowed to use any of four procedures for determining the sales proportions (U.S. BLS, 1997): obtaining the proportions directly from a respondent; ranking the subgroups/items by importance of sales as indicated by the respondent and then obtaining the proportions directly or using pre-assigned propor- tions; using shelf space to estimate the proportions where applicable; using equal probability.
5.25 The advantages of this procedure, according to the BLS, are that it ensures an objective and eﬃcient probability sampling procedure, where no other such procedure would be available.

It allows broad deﬁni- tions of the item strata so that the same tight speciﬁca- tion need not be priced everywhere.
The wide variety of speciﬁc items greatly reduces the within-item component of variance, reduces the correlation of price movement between areas, and allows a reduction of the sample size needed for a given variance.

5.26 A potential pitfall in this approach is that, if the sales value measure is taken during a very short period, it may coincide with a special campaign with temporarily reduced prices.
It could then happen that an item with a temporarily reduced price is given a large inclusion probability.

Since this price will tend to increase more than average, an overestimating bias may result.

It is thus essential that the sampling of the item takes place at an earlier point in time than the ﬁrst price collection or that sales values from an earlier period are used.

Okamoto (1999) emphasizes this point for Japan, where price bouncing seems to be a very common phenomenon.
Non-probability sampling techniques 5.27 Modern statistical sampling theory focuses on probability sampling.

Use of probability sampling is also strongly recommended and standard practice for all kinds of statistical surveys, including economic surveys.

But price index practice in most countries is still domi- nated by non-probability techniques.

It may then be fruitful to speculate somewhat about the rational and irrational reasons for this situation.
In the following section we discuss a number of such possible reasons, one by one.

We then go on to consider some non- probability techniques.

Reasons for using non-probability sampling 5.28 No sampling frame is available.

This is often true for the product dimension but less frequently so for the outlet dimension, for which business registers or telephone directories do provide frames, at least in some countries, notably in Western Europe, North America and Oceania.
There is also the possibility of constructing tailor-made frames in a limited number of cities or loca- tions, which are sampled as clusters in a ﬁrst stage.

For products, it may be noted that the product assortment 71 CONSUMER PRICE INDEX MANUAL: THEORY AND PRACTICE exhibited in an outlet provides a natural sampling frame, once the outlet is sampled as a kind of cluster, as in the BLS sampling procedure presented above.
So the absence of sampling frames is not a good enough excuse for not applying probability sampling.

5.29 Bias resulting from non-probability sampling is negligible.

There is some empirical evidence to support this assertion for highly aggregated indexes.

Dale´ n (1998b) and De Haan, Opperdoes and Schut (1999) both simulated cut-oﬀ sampling of products within item groups.
Dale´ n looked at about 100 groups of items sold in supermarkets and noted large biases for the sub- indices of many item groups, which however almost cancelled out after aggregation.

De Haan, Opperdoes and Schut used scanner data and looked at three cate- gories (coﬀee, babies’ napkins and toilet paper) and, although the bias for any one of these was large, the mean square error (deﬁned as the variance plus the squared bias) was often smaller than that for pps sampling.
Biases were in both directions and so could be interpreted to support Dale´ n’s ﬁndings.

The large biases for item groups could, however, still be disturb- ing.

Both Dale´ n and De Haan, Opperdoes and Schut report biases for single-item groups of many index points.

5.30 We need to ensure that samples can be monitored for some time.

If we are unlucky with our probability sample, we may end up with a product that disappears immediately after its inclusion in the sample.
We are then faced with a replacement problem, with its own bias risks.

Against this, it may happen that short-lived pro- ducts have a diﬀerent price movement from the price movement of long-lived ones and constitute a signiﬁcant part of the market, so leaving them out will create bias.

5.31 A probability sample with respect to the base period is not a proper probability sample with respect to the current period.

This argument anticipates some of the discussion in Chapter 8 below.
It is certainly true that the bias protection oﬀered by probability sampling is to a large extent destroyed by the need for non-probabil- istic replacements later on.

5.32 Price collection must take place where there are price collectors.

This argument applies to geographical sampling only.

It is, of course, cheaper to collect prices near the homes of the price collectors, and it would be diﬃcult and expensive to recruit and dismiss price col- lectors each time a new sample is drawn.
This problem can be reduced by having good coverage of the country in terms of price collectors.

One way to achieve this is to have a professional and geographically distributed interviewer organization within the national statistical agency, which works on many surveys at the same time.

Another way of reducing the problem is to have a ﬁrst- stage sample of regions or cities or locations which changes only very slowly.

5.33 The sample size is too small.


Stratiﬁcation is sometimes made so ﬁne that there is room for only a very small sample in the ﬁnal stratum.

A random selection of 1–5 units may sometimes result in a ﬁnal sample that is felt to be skewed or otherwise to have poor representa- tivity properties.

Unless the index for this small stratum is to be publicly presented, however, the problem is also 72 small.

The skewness of small low-level samples will even out at higher levels.
The argument that sample size is too small has a greater validity when it relates to ﬁrst-stage clusters (geographical areas) that apply to most sub- sequent sampling levels simultaneously.

5.34 Sampling decisions have to be taken at a low level in the organization.

Unless price collectors are well versed in statistics, it may be diﬃcult for them to perform probability sampling on site.
Such sampling would be necessary if the product speciﬁcation that has been pro- vided centrally covers more than one product (price) in an outlet.

Nevertheless, in the United States (U.S. BLS, 1997) ﬁeld representatives do exactly this.

In Swe- den, where central product sampling (for daily neces- sities) is carried to the point of specifying well-deﬁned varieties and package sizes, no sampling in the outlets is needed.
In countries where neither of these possibilities is at hand, full probability sampling for products would be more diﬃcult.

5.35 In some situations, there are thus valid reasons for using non-probability techniques.

We discuss two such techniques below.

Cut-off sampling 5.36 Cut-oﬀ sampling refers to the practice of choosing the n largest sampling units with certainty and giving the rest a zero chance of inclusion.
In this context, the term ‘‘largeness’’ relates to some measure of size that is highly correlated with the target variable.

The word ‘‘cut-oﬀ ’’ refers to the borderline value between the included and the excluded units.

5.37 In general, sampling theory tells us that cut-oﬀ sampling does not produce unbiased estimators (see paragraphs 5.51 to 5.60 below for a discussion of bias and variance), since the small units may display price movements which systematically diﬀer from those of the larger units.
Stratiﬁcation by size or pps sampling also has the advantage of including the largest units with certainty while still giving all units a non-zero prob- ability of inclusion.

5.38 If the error criterion is not minimal bias but minimal mean square error (=variance+squared bias) then, since any estimator from cut-oﬀ sampling has zero variance, cut-oﬀ sampling might be a good choice where the variance reduction more than oﬀsets the introduc- tion of a small bias.
De Haan, Opperdoes and Schut (1999) demonstrate that this may indeed be the case for some item groups.

5.39 Often, in a multi-stage sampling design there is room for only a very small number of units at a certain stage.

Measurement diﬃculties that are sometimes asso- ciated with small units may then be a reason, in addition to large variances, for limiting price collection to the largest units.
5.40 Note that a hybrid design can also be applied in which there is a certainty stratum part, some probability sampling strata and a low cut-oﬀ point below which no sample at all is drawn.

In practice, this design is very often used where the ‘‘below cut-oﬀ section’’ of the universe is considered insigniﬁcant and perhaps diﬃcult to measure.
5.41 A particular CPI practice that is akin to cut-oﬀ sampling is for the price collector to select the most sold product in an outlet, within a centrally deﬁned speciﬁ- cation.

In this case, the sample size is one (in each outlet) and the cut-oﬀ rule is judgemental rather than exact, since exact size measures are only rarely available.
In all cases of size-dependent sampling in an outlet, it is cru- cial to take a long-term view of size, so that temporarily large sales during a short period of reduced prices are not taken as a size measure.

Such products will tend to increase in price in the immediate future much more than the product group which they represent and thus create a serious overestimating bias.
Quota sampling 5.42 Many product groups, even rather small ones, are quite heterogeneous in nature, and the price varies according to a large number of subgroups or char- acteristics.

There may well be diﬀerent price movements going on within such a product group, and a procedure to represent the group by just one or a few tightly spe- ciﬁed product types may then carry an unnecessarily great risk of bias.
5.43 The deﬁnition of quota sampling is that the selected sample shall have the same proportions of units as the universe with respect to a number of known characteristics, such as product subgroup, type of outlet, and location.

The actual selection of sampling units is then done by judgemental procedures in such a manner that the composition of the ﬁnal sample meets the quota criteria.

5.44 The following example illustrates the concept of quota sampling.

A sample of 20 package holidays is desired.


It is known that, in the universe, 60 per cent of the holidays are to Spain, 30 per cent to Greece, and 10 per cent to Portugal.

Of the travel groups, 70 per cent comprise 2 adults, 20 per cent comprise 2 adults +1 child, and 10 per cent comprise 2 adults +2 children.

Of the sample, 20 per cent stay in 2-star hotels, 40 per cent in 3-star hotels, 30 per cent in 4-star hotels, and 10 per cent in 5-star hotels.
With this information, it is possible to design the sample purposively so that all these propor- tions are retained in the sample, which then becomes self- weighted.

Note that these proportions reﬂect volumes, not values, and may need to be adjusted depending on the elementary aggregate formula used.

5.45 Quota sampling requires central management of the whole sampling process, which may limit its use- fulness in some situations.
It is more diﬃcult, but not impossible, to manage a quota sampling system where local price collection is used.

One would then need to divide the price collectors into subgroups with somewhat diﬀerent instructions for selecting products.

A limitation of quota sampling, as in other non-probability sampling, is that the standard error of the estimate cannot be determined.

The representative item method 5.46 This is the traditional CPI method.
The central oﬃce draws up a list of product types, with product-type speciﬁcations.

These speciﬁcations may be tight, in that SAMPLING they narrowly prescribe for the price collectors what products they are permitted to select, or they may be loose, giving the price collector freedom to choose locally popular varieties.

5.47 The method with tight speciﬁcations is in a sense diametrically opposite to the quota sampling method discussed above.
Unless the product groups are deﬁned so as to include a very large number of product types, representativity will suﬀer in this procedure, since no products falling outside the speciﬁcation will enter the index.

Another disadvantage with the method is that it may lead to more missing products in the outlets and thus reduce the eﬀective sample.

Its main advantage is simplicity.

It is easy to maintain a central control over the sample.
If quality adjustments are needed, they can be decided in the central oﬃce, which may or may not be an advantage.

5.48 The method with loose speciﬁcations gives price collectors the chance to adjust the sample to local con- ditions and will normally lead to greater representativity of the sample as a whole.

Where it is combined with the ‘‘most sold’’ criterion it will, however, systematically underrepresent the smaller brands and products that may be bought by important minorities.
Sampling in time 5.49 A CPI usually refers to a month, during which prices are not constant.

The issue of sampling in time then arises.

Often, this problem is ignored, for example by using the 15th day of the month, or the days sur- rounding the 15th, as the target date for price measure- ment.
In some areas, there is a day-of-the-week eﬀect on prices, for example in cinemas, theatres and restaurants, but this may be taken into account in the product spe- ciﬁcation rather than in sampling, for example by spe- cifying a weekday evening price.

5.50 As far as is known, random sampling in time is not used anywhere.
The method used in some countries is to spread price collection over several weeks according to some pattern, for example diﬀerent weeks in diﬀerent regions or for diﬀerent product groups.

In some cases, more frequent pricing than monthly is also used, for example for fresh produce.

There is not yet any sys- tematic knowledge about the pros and cons of such practices.

Chapter 6 discusses the more practical aspects of distributing price collection over time.
Choice of sampling method 5.51 In this section, we discuss how choices in sam- pling method could depend on speciﬁc factors in a country.

But ﬁrst we consider the matter of sample size.

5.52 Sample size.

The ﬁnal precision of a sample estimate depends only on the size and allocation of the sample and not on the size of the country, so in this sense there is no need for a larger sample in a larger country.
Larger samples are called for if regional dif- ferences in price change are of interest and if the amount of product disaggregation that is desired in presenting the indices is very high.

Of course, the budget allocated 73 CONSUMER PRICE INDEX MANUAL: THEORY AND PRACTICE to CPI work may be larger in large countries, allowing for larger samples.
5.53 Studies of bias (not the sampling bias described in paragraphs 5.61 to 5.64) and of sampling error show that bias in CPIs is generally a much greater problem than sampling error.

This leads to the conclusion that, in many cases, smaller samples that are better monitored with respect to replacements, resampling and quality adjustment could give a higher quality index for the same budget.
In some countries, local price collection is a ﬁxed resource and it is therefore diﬃcult to move resources from local price collection to central analytical work.

Still, it is advisable to try to use local resources for higher quality price collection rather than just for many observations.

The quality of price collection is further discussed in Chapter 6.

5.54 Monthly sample sizes in diﬀerent countries seem to vary from several thousand to several hundred thousand.
Often, the reasons for these diﬀerences lie more in tradition than in a rational analysis of the needs of precision.

Countries with very large sample sizes would probably do well to look at ways of reallocating their total resources.

5.55 Geographical distribution of price collectors.

Sampling is more expensive further away from the homes of the price collectors.

If the organization for price collection is centralized in a few main cities, it will be diﬃcult to sample outlets elsewhere.
It should be borne in mind, however, that rural and urban inﬂation may well be diﬀerent, so failure to collect prices in both rural and urban areas would be detrimental to eﬀorts to achieve the best measure of average national inﬂation.

It would be better to have at least a small sample in the rural areas so that this factor can be taken into account.

The major part of the saving arising from allocating outlets close to price collectors can then still be realized.

5.56 Sophistication of price collectors.


If price col- lectors are well educated, they may be instructed to carry out more complex sampling schemes such as pps sampling in the outlets.

Otherwise, simpler methods are called for.

5.57 Access to sampling expertise in the central oﬃce.

Probability sampling requires access to methodological expertise in the central statistical oﬃce.

5.58 Homogeneous versus heterogeneous product groups.
The representative item method is more suitable for homogeneous product groups.

In heterogenous groups, it is more likely that important segments of the product universe, with diﬀerent price movement, will be left out.

5.59 Access to sampling frames and their quality.

Probability sampling requires sampling frames.

But they do not necessarily have to be available at the national level.
By applying geographical cluster sampling in the ﬁrst stage (where the sampling frame is just a map), a list of relevant outlets can be constructed in each sampled cluster using telephone directories or local enumeration, as is done in the United Kingdom.

This method is also used to select urban areas for the United States CPI (Dippo and Jacobs, 1983).

5.60 Scanner data.
5.60 Scanner data.

The discussion in this chapter is based on the traditional situation, where prices have to 74 be collected locally and centrally, and entered indivi- dually into a central database.

Where prices and possi- bly quantities are collected electronically, as is the case with sale point scanner data, sampling could be diﬀer- ent.

There is then no need for sampling products or varieties or points in time, since they are completely enumerated automatically.
Nevertheless, not all outlets selling a product will be covered by scanner data in the foreseeable future.

Since all kinds of outlets should be represented in the index, there will be continue to be a need to combine scanner data samples with traditional samples for non-scanner outlets.
Estimation procedures 5.61 There is a crucial distinction to be made between what is to be estimated, the parameter, which is deﬁned for the whole universe, and the estimator, which is a formula to be calculated using the sample values as an estimate of the parameter.

Now, in survey sampling in general we want to estimate a population total or a function of several such totals, for example a ratio of totals.
So, if we have two variables y and z deﬁned for each sampling unit (for example, prices at two diﬀerent periods), we may want to estimate the following pa- rameters: N Y= P j=1 yj N and Z= P j=1 zj or R=Y=Z 5.62 Several diﬀerent estimators may be proposed for the same population parameter, in which case we need to decide which of these estimators to use.

In assessing the quality of a sample estimator, i.e.
how well it estimates the parameter, two measures are often con- sidered in the probability sampling paradigm.

The ﬁrst measure is the bias of an estimator, which is the diﬀer- ence between the universe parameter and the average of the estimator over all possible samples that could be drawn under the speciﬁed sample design (referred to as the mean of the sampling distribution of the estimator).

Note that this bias refers to something diﬀerent from the index number bias discussed elsewhere in this manual.
An estimator is unbiased if it has zero bias.

The second measure is the variance of the estimator with respect to this sampling distribution.

An estimator is considered good if both its bias and variance are small; that is, the estimator is on average very close to the parameter and does not vary much from its mean.

5.63 The good fortune of ﬁnding an estimator that minimizes both bias and variance at the same time does not often happen.
An estimator with a small bias may have a large variance, and one with a small variance may have a large bias.

So use is frequently made of a criterion called the mean square error, which is the sum of the bias squared and the variance.

A ‘‘good’’ estimator is then one which minimizes this criterion.
5.64 Sampling theory tells us that the following estimators are unbiased, respectively, for the parameters Y and Z above: ^YY=P j 2 Szj=pj, where S is the sample, and that ^RR= ^YY= ^ZZ is approximately j 2 Syj=pj, ^ZZ=P unbiased for R, subject to a (usually negligible) technical ratio estimator bias.

Implementing estimation procedures for consumer price indices 5.65 As stated earlier, sampling for CPIs is usually stratiﬁed, with elementary aggregates as strata.
Let us assume that the universe parameter is I and that the parameter in stratum h is labelled Ih.

Then we have: I=P h whIh where wh is the weight of stratum h. The issue then is to estimate Ih for each stratum.
In the following discussion, we therefore concentrate on estimating for a single stratum and drop the subscript h. 5.66 Depending on the content, degree of homo- geneity, price elasticity and access to weighting infor- mation within the stratum, diﬀerent parameters may be appropriate in diﬀerent strata.

The choice of parameter is an index number problem, to be solved by reference to the underlying economic concepts.
As discussed in Chapter 20, it could be the unit value index, the Laspeyres index, the Lowe index, or the geometric Laspeyres index.

5.67 Suppose we have a sample of size n and that the units in the sample are labelled 1,2, .

.

.
.

.

, n. Very often, one of the three formulae below is used as an estimator of the stratum index: The arithmetic mean of price relatives (Carli index): r= 1 n P j 2 S p1 j p0 j The ratio of mean prices (Dutot index): a= 1 n 1 n P j 2 S P j 2 S p1 j p0 j The geometric mean (Jevons index): g= n !1 p1 j p0 j Y j 2 S (5.1) (5.2) (5.3) For discussion below, we also need to introduce the ratio of harmonic mean prices: P j 2 S P j 2 S 1=p0 j 1=p1 j (5.4) h= 1 n 1 n SAMPLING Balk (2002) for details).
Suppose we have N products in the universe labelled 1,2, .

.

.

, N. Let pt j be respectively the price and quantity for product j in period t (t=0 for base period and 1 for current period), and let j; qt w0 j = j p0 q0 j N P j=1 j p0 q0 j ( j=1, .

.

.

, N) be the base period expenditure share of product j.

Then: Under simple random sampling, none of r, a or g estimates any of the population parameters without bias.

Instead, weights need to be used in the estima- tors also.
Under pps, if pj / w0 j for all j, then r, the average of is unbiased for the Laspeyres index (the relatives, symbol ‘‘/’’ means ‘‘proportional to’’).

 Under pps, if pj / q0 j for all j, then a, the ratio of averages, is approximately unbiased for the Laspeyres index.

 Under pps, if pj / w0 j for all j, then g is approximately unbiased for the geometric Laspeyres index.

In this case log g is unbiased for the logarithm of the geo- metric Laspeyres index.
The remaining bias tends to be of a similar order to that of a.

5.70 All these results are somewhat theoretical in nature since neither w0 j nor q0 j are known at the time when the sample could be drawn.

This is a reason for introducing the Lowe index: Under pps, if pj / qb j (where b is some period before 0) for all j, then a is approximately unbiased for the Lowe index.

5.71 There is no simple way to relate any of the estimators to the unit value index.
In fact, estimating that index requires separate samples in the two time periods, since its numerator and denominator refer to diﬀerent universes.

 Under two separate sample designs, one for period 0 and one for period 1, which are both pps and where p0 j / q0 j , then a is approximately unbiased for the unit value index.

In this case, however, the interpretation of the a formula will be diﬀerent, since the samples in the numerator and the denominator are diﬀerent.
j and p1 j / q1 j q0 j =p0 Under two separate sample designs, one for period 0 and one for period 1, which are both pps and where p0 j =p1 j / v0 j , then h, the ratio of harmonic mean prices, is approximately unbiased for the unit value index.
The following algebraic refor- mulation of the unit value index helps to clarify that fact: j and p1 j / v1 j q1 5.68 When comparing the above estimators with the functional form of the parameters in Chapter 20, we realize that very special conditions are needed to make them unbiased estimators of those parameters.

For one thing, unlike the parameters in Chapter 20, there are no quantities involved in the sample estimators.
5.69 We state, without proof, some results concern- ing the statistical properties of the above estimators (see UV= P j 2 S P j 2 S v1 j v0 j .

P j 2 S .

P j 2 S j =p1 v1 j j =p0 v0 j : As for a, however, the interpretation of the h formula will be diﬀerent, since the samples in the numerator and the denominator are diﬀerent.

75 CONSUMER PRICE INDEX MANUAL: THEORY AND PRACTICE 5.72 The phrase ‘‘approximately unbiased’’ needs some explanation.
It refers to the fact that the estimator is not exactly unbiased but that the bias is small and decreases towards zero as the sample size and the size of the universe simultaneously go to inﬁnity in a certain, mathematically well-deﬁned manner.

In the ratio esti- mator case applicable to a, the sign of this bias is in- determinate and its size after aggregation is probably negligible.

In the case of the geometric mean, however, the bias is always positive, i.e.
the sample geometric mean tends to overestimate the universe geometric mean on average over many sample drawings.

In the case of sim- ple random sampling and an unweighted geometric mean in both the universe and the sample, the bias expression is: b&s2/2n, where s2 is the variance of the price ratios.

For small universes, a ﬁnite population correction needs to be multiplied to this expression.

This result is easily derived from expression (4.1.4) in Dale´ n (1999b).
The bias may be signiﬁcant for small sample sizes, so that a caution against very small samples in a stratum may be warranted when the geometric mean is applied.

Variance estimation 5.73 A CPI is a complex statistic, usually with a complex design.

It is thus not a routine task to estimate the variance of a CPI.

To the extent that samples are not probability based, variance estimates need to make use of some kind of model in which random sampling is assumed.
In the absence of systematic and generally accepted knowledge, the approaches to variance esti- mation used in four countries will be brieﬂy described.

Variances of elementary index formulae 5.74 As a preliminary, some variance estimators for elementary aggregate formulae will be provided.

In order not to overburden the text with formulae, the variance estimators, not the exact variance, will be given.
The variance estimators are approximately unbiased under simple random sampling, where the corresponding uni- verse parameter is unweighted.

They are also applicable to the case of pps sampling for a weighted universe parameter, where the size measure is the same as the parameter weight.

For deﬁnitions of the formulae, see equations (5.1)–(5.3).
V(r)= s2 r n , where s2 r = and rj= (rj r)2 P j 2 S 1 n 1 p1 j p0 j ; (5.5) (5.6) V(a)= 0 2rs01), 1+r2s2 1 n( pp 0)2 (s2 j pp1)2, s2 j 2 S ( p1 j pp 0), j pp1)( p0 P j 2 S p0 j .

P 0= 1 n1 P 1= 1 n1 P j 2 S ( p1 j 2 S p1 where s2 s01= 1 n1 P pp1= 1 n This estimate follows from the fact that a, unlike r, is a ratio of stochastic variables.
See, for example, Cochran (1977) for a derivation of this formula.

j and pp 0= 1 n j pp 0)2, j 2 S ( p0 76 5.75 The geometric mean is more complex, since it is not a linear estimator.
However, Dale´ n (1999b) derived the following easily applied variance expression, which holds with good approximation if price ratios do not have too extreme variation (sr/r<0.2, say): V(g)= s2 r n 1 s2 r r2 (5.7) The United States approach 5.76 The United States CPI uses sampling and esti- mation procedures which are in many ways unique in comparison to those of other countries.

The exact design obviously varies somewhat over time.
The following description is based on U.S. BLS (1997) and Leaver and Valliant (1995).

5.77 The United States CPI is composed of building blocks consisting of geographical areas crossed with product strata to a total of 8,487 ‘‘basic CPI strata’’ corresponding to elementary aggregates.

The 88 geo- graphical areas were selected by pps in a controlled selection procedure and 29 of them were included with certainty (self-representing).
Within each basic CPI stratum an estimation procedure is applied in which indices for a particular time period are based on the overlapping sample units (outlets and items) between this time period and the immediately preceding period.

The period-to-period indices are then multiplied to obtain an index from the base period to the current period.

Sam- pling within the basic CPI strata is approximately pps according to the description above.
5.78 Variance estimation for this design proves to be too complex for the use of a direct design-based variance estimator.

Instead a random group replication method, using the so-called VPLX software, is applied.

Other methods have also been tried.

5.79 Leaver and Swanson (1992) provide a detailed account of the variance estimation methods used up to then.
They also present the following numerical esti- mates of (median) standard errors for CPI changes for various intervals over the 1987–91 period: 1 month standard error 0.074; 2 months standard error 0.103; 6 months standard error 0.130; and 12 months standard error 0.143.

The Swedish approach 5.80 The following outline summarizes the descrip- tion given by Dale´ n and Ohlsson (1995).
The Swedish CPI uses a primary stratiﬁcation into product groups, which are measured in separate and independent price surveys.

The ﬁrst step in the Swedish approach is there- fore to note that the variance of the all items price index is a weighted sum of the variances of the separate sur- veys: V(I)=P h w2 hV(Ih) (5.8) 5.81 The reason that all these surveys can reasonably be assumed to be independent is that there is no common regional sampling scheme used in them.
Altogether, there are about 60 diﬀerent surveys.

Some of them cover many product groups and have a complex design and there is stochastic dependence between them.

Other surveys cover only one group and have simple designs.

Some cover their universes, without any sampling, so they have zero variance.

5.82 In many simple product groups it is fairly rea- sonable to assume that the price ratios obtained are eﬀectively random samples.
In some cases this may lead to some overestimation of variance since there is in fact some substratiﬁcation or quota sampling within the group.

In those product groups, stratum variances could then be estimated according to formulae (5.5)–(5.7).

When a price survey is stratiﬁed, formula (5.8) can be applied at lower levels above the elementary aggregate.

5.83 Some price surveys are more complex, how- ever.
This is especially the case for that large part of the index where outlets and products are simultaneously sampled.

In the Swedish case these surveys are called the local price survey and the daily necessities survey.

In both these cases, outlets are sampled by probability (pps) from the central business register.

Products are sampled by pps in the daily necessities survey but by the representative item method in the local price survey.
In the Swedish variance estimation model, the ﬁnal sample is in these cases considered as drawn from a two- dimensional universe of products and outlets.

The ﬁnal sampling units are thus sampled products sold in sam- pled outlets – a cross-classiﬁed sample.

5.84 In a cross-classiﬁed sample, the total variance can be decomposed into three parts: – variance between products (in the same outlet); – variance between outlets (for the same product); – outlet and product interaction variance.
Dale´ n and Ohlsson (1995) provide the exact formulae used.

5.85 In the daily necessities survey, the cross-classi- ﬁed model comes fairly close to the actual sampling design.

In the local price survey, it is more of a model, since the products are in fact purposively drawn.

It has nevertheless been considered a useful model for the purpose of getting a ﬁrst idea of the sampling error and for analysing allocation problems.
5.86 The total variance of the Swedish CPI, accord- ing to this model, was estimated to be 0.04, corre- sponding to a 95 per cent conﬁdence interval of ±0.4.

This estimate appeared to be fairly stable over the period 1991–95 for which the model was tried.

The French approach 5.87 In France, variance calculation at present only takes into consideration items accounting for 65 per cent of the total weight of the index.

5.88 The smallest element of the calculation is a product type in an urban area.
For these elements one of two formulae are applied, depending on whether the product is homogeneous (ratio of arithmetic means) or heterogeneous (geometric means).

A two-stage random sample is assumed, ﬁrst of urban areas and then of a particular item (variety) in an outlet.

The variance SAMPLING obtained is thus the sum of a ‘‘between urban areas’’ and a ‘‘within urban areas’’ component.

Linearization based on second-degree expansions is done, because of the non-linear nature of the estimators.
Higher-level vari- ances are obtained by weighting the elementary level variances.

5.89 After an optimization exercise which took place in 1997, the standard deviation of the all-products index (for 65 per cent of the total weight of the index) was computed as 0.03.

This value is close to that estimated in 1993, although the number of observations was reduced.

The precision of a number of sub-indices was also improved.

5.90 Covariance terms are ignored.


In fact, this makes a very small diﬀerence in the ‘‘between urban areas’’ component.

In the ‘‘within urban areas’’ compo- nent it has undoubtedly a greater inﬂuence.

The eﬀect is, however, seen as limited because of a rule which limits the number of products observed in the same outlet.

5.91 For the 35 per cent of the weight that is at present excluded from the variance calculation (called the ‘‘tariﬀs’’), such calculations will be introduced for insurance.
The necessary elements for variance cal- culation are also present for physicians’ and dentists’ services.

Variances will soon be calculated for these products, as well as for new cars.

For a certain number of sub-indices (tobacco and pharmaceuticals) the sample is in eﬀect a total count.

Their variances are thus zero.

5.92 A 95 per cent conﬁdence interval for a 12-month comparison can be expressed as the estimated index ±0.06 for the ordinary, non-tariﬀ items.
If zero variance is assumed for the remaining 35 per cent of the index, the conﬁdence interval for the all-products index would become ±0.04.

This assumption is clearly too optimistic, but from the work on variance estimation done so far, it can be concluded that the conﬁdence interval is certainly smaller than 0.1.

5.93 More details on the French computations can be found in Ardilly and Guglielmetti (1993).
The Luxembourg approach 5.94 The Luxembourg CPI can be described as a stratiﬁed purposive sample with 258 product strata.

There are slightly fewer than 7,000 observations each month, giving an average of 27 observations per stra- tum.

In each stratum, observations are taken from sev- eral diﬀerent outlets; but the same outlet is represented in many product strata.
The outlet is here used as the identiﬁer for the price-setting organization (for rents it is a landlord, for insurance it is the companies, and so on).

In each stratum, there are observations from several outlets.
Since there is good reason to believe that each outlet has its own price-setting behaviour, prices and price changes in the same outlet tend to be correlated, resulting in positive covariances in the general variance expression: V(I)=P kw2 kV(Ik)+P P k lwkwlCov(Ik; Il) (5.9) 5.95 In the sampling model, each separate outlet sample within a product stratum is regarded as a simple random sample.
Further, a two-stage model was 77 CONSUMER PRICE INDEX MANUAL: THEORY AND PRACTICE assumed such that, in the ﬁrst stage, a simple random sample of outlets was assumed to have been drawn from a (ﬁctitious) sampling frame of all outlets in Lux- embourg.

Then, in each sampled outlet, a second-stage sample of observations was assumed to be drawn in product stratum h so that the combined product–outlet stratum became the lowest computational in the index.
All second-stage samples were assumed to be mutually independent and sampling fractions to be small.

This model resulted in three components of total variance: variance within outlets; variance between outlets; covariance between outlets.

Covariances are diﬃcult to calculate, even with a com- puter.

Luckily, however, it was possible algebraically to combine the last two components into one, with the number of summation levels reduced.
level 5.96 Numerical estimates were made with this model for 22 consecutive 12-month changes starting from the period January 1996 to January 1997 and ending with the period October 1997 to October 1998.

The average variance estimate was 0.02 (corresponding to a standard error of 0.14), which is surprisingly small given the small sample size.
The reason for this small value was not explored in detail but may lie in a combination of the special circumstances in the markets in Luxembourg and in procedures used in the index estimation system.

5.97 The full variance estimation model for the Luxembourg CPI and the results from it are presented in Dale´ n and Muelteel (1998).

Other approaches 5.98 A number of experimental models have been tried out and calculations done for the United Kingdom.
None of them has so far been acknowledged as an oﬃcial method or estimate.

Kenny (1995 and earlier reports) experimented with the Swedish approach on United Kingdom data.

He found a standard error of the United Kingdom Retail Price Index as a whole of around 0.1, which was reasonably constant over several years, although the detailed composition of the variance varied quite a lot.

Sitter and Balshaw (1998) used a pseudo-population approach but did not present any overall variance estimates.
5.99 For Finland, Jacobsen (1997) provided partial calculations according to a similar design as in the Swedish approach.

His analysis was used to suggest changes in the allocation of the sample.

for the variance of the estimate and another expression for the cost.

Both variance and cost are functions of sample size.

Optimal allocation then amounts to mini- mizing variance for a given cost or minimizing cost for a given variance.

5.102 Variance estimation was discussed above.


As for cost, it is important to note that not all price observations are equally costly.

It is less expensive to collect an extra price in an outlet that is already in the sample than to add a price in an outlet that is new to the sample.
For example, in the Swedish CPI, the following cost function was used: C=C0 þ P n hnh ah+bh o P gmgrgh (5.10) where C refers to total cost and C0 to the ﬁxed part of the cost that is independent of sample size, nh is the number of outlets in outlet stratum h, mg is the number of product varieties in product stratum g, ah is the unit cost per outlet and reﬂects travelling time to the outlet, bh is the unit cost per product, which reﬂects the addi- tional cost for observing a product, when the price collector is already in the outlet, rgh is the average relative frequency of products in stratum g sold in outlets of stratum h. 5.103 In formula (5.10), ah is usually much larger than bh.
This fact calls for an allocation with relatively more products than outlets, i.e.

of several products per outlet.

This allocation is further reinforced to the extent that variances between products in the same outlet and product stratum are usually larger than variances between outlets for the same product.

This is the case, at least according to experience in Sweden.
5.104 With a speciﬁed variance function and a spe- ciﬁed cost function, it is possible, using the mathematical technique of Lagrange multipliers, to derive optimal sample sizes in each stratum.

It is usually not possible to obtain explicit expressions, however, since we run into a non-linear optimization problem for which it is not possible to ﬁnd an explicit solution.

5.105 In a CPI, the all-products index is usually the most important statistic.
Therefore, the allocation of the sample should be directed towards the minimization of its error.

It is also important that other published sub- indices are of high quality, but the sub-index quality can often be taken as the criterion for publication, rather than the other way round.

Summary Optimal allocation 5.100 Producing a consumer price index is a major operation in any country and a great deal of resources are spent on price collection.
It is therefore worthwhile to devote some eﬀort to allocating these resources in the most eﬃcient way.

5.101 The general approach to sample allocation was established by Neyman and is described in any sampling textbook.

It uses a mathematical expression 5.106 The above discussion can be summarized in the form of a small number of speciﬁc recommenda- tions.

5.107 Clarity – sampling rules should be well deﬁned.
In many CPIs, there is a wide range of sampling and other solutions for diﬀerent product groups.

A fairly well-deﬁned method is often used for the ﬁeld collection of prices, but the exact methods used for the central price collection of many products are commonly in the hands of one or a few responsible persons and are 78 SAMPLING sometimes poorly documented.

It is essential for the basic credibility of the CPI that rules for sampling and estimation (e.g.
the treatment of outliers) are well deﬁned and described.

5.108 Probability sampling should be seriously con- sidered.

The use of probability sampling designs should be increased.

In many areas, useful sampling frames do exist or could be constructed without excessive diﬃcul- ties.

Stratiﬁed, order pps sampling is an important type of design that ought to be considered in many situations.
Size measures used for sampling must have a long-term interpretation, so that they are uncorrelated with price movements.

5.109 Representativity – no large part of the universe should be left out.

When sampling designs are planned, the full universe of items and outlets belonging to the item group in question should be taken into account.

All signiﬁcant parts of that universe should be appropriately represented, unless there are excessive costs or estima- tion problems involved in doing so.
5.110 Variance or mean square error should be as low as possible.

Samples should be reasonably optimized, based on at least a rudimentary analysis of sampling variance.

As a ﬁrst-order approximation, sample sizes could be set approximately proportional to the weights of the commodity groups.

A better approximation is obtained by multiplying each weight by a measure of price change dispersion in the group.
Variance and cost considerations together call for allocations where rela- tively many products are measured per outlet and rela- tively few outlets are contained in the sample.

Since biases are generally a greater problem than sampling errors, smaller but better samples, allowing for more frequent renewal and careful monitoring of replace- ments and quality adjustments, generally make good sense.
79 PRICE COLLECTION Introduction 6.1 The most appropriate sampling and survey methods for a price survey will vary depending on the use of the price index and local circumstances.
For instance, the diversity of available goods and services, their turnover and the range of prices charged, the fre- quency and size of price changes, consumers’ purchasing habits (including the use of telephone, catalogue and Internet shopping) and the structure of retailing in terms of the local economy, types of outlets and geographical spread will all have a bearing.
6.2 This chapter gives an overview of some of the issues, but bearing the above in mind it is clear that the treatment of these will require diﬀerent solutions in diﬀerent countries according to local circumstances.

Solutions cannot be too prescriptive, and the compiler should always be guided by the fundamental principles and objectives of a price index as addressed in earlier chapters.
The structure of Western economies together with the retailing patterns and associated consumer purchasing habits lend themselves to more structured price collections.

In contrast, subsistence economies and developing economies will require more ﬂexible price collection techniques.

6.3 Consideration has to be given as to how best to collect prices in terms of eﬃciency, accuracy and representation of consumers’ purchasing patterns.
In some cases, price collection directly from individual shops around the region or country (local price collection) may be considered appropriate.

In other collected centrally by staﬀ circumstances, prices located in the headquarters or regional oﬃces of the national statistical institute (central price collection) may be more appropriate.

Many of the issues covered in this chapter are relevant to both local and central price collection.
6.4 The advantages and disadvantages of local ver- sus central price collection for diﬀerent types of prices are covered later in this chapter.

Brieﬂy, local price collection has the advantage of covering a wide range of locations and item selections, particularly for food, alcohol, tobacco and durable goods (for example, clothing, furniture and electrical goods).
Central col- lection is useful for prices that are diﬃcult to observe directly (for example, costs of housing or utilities), where there are national pricing policies, for goods sold through mail order and catalogues, or for items where there are limited collection opportunities or diﬃculties in making an adjustment for technical and quality charges (particularly transport and services).
6 Frequency and timing of collection 6.5 The type of economy can initially govern the frequency and timing of price collection.

Where tran- sient markets are important to a wide spectrum of the population, the timing of these markets will aﬀect the timing of price collecting because of the need to consider the availability of goods and services to consumers.
6.6 A fundamental decision about the frequency and timing of price collection is whether the index should relate to monthly average prices or prices for a speciﬁc point in time (for example, a single day or week in a month).

This decision is related to a number of factors, including the uses of the index, the practicalities of carrying out price collections, the pattern of price movements and the timing of index publication.

These factors are discussed in turn below.


6.7 It has been argued that the question of whether the index should relate to a period or to a point in time generally becomes less of an issue the more frequently the prices are collected.

But it is far from certain whether this is true in all circumstances.

For instance, prices over certain holiday periods or at particular times of year might be especially volatile.
In such cases, the smooth- ness resulting from a period rather than a point estimate may be considered an advantage over a potentially misleading short-term trend shown by a more highly volatile point estimate.

In answering this question, there is also a need to take into account the primary use of the index.

6.8 In principle, if used for deﬂating income, expen- diture or sales, the index should relate to the period of time of these money ﬂows.
For economic analysis, where the index will be used in conjunction with other economic statistics, most of which relate to a period rather than to a point in time, it seems logical, again in principle, that the consumer price index should do the same.

6.9 In reality, when making this choice, consider- ations of principle have to be weighed against various practical considerations.
The ﬁrst point to note is that when inﬂation is low and stable there will be little dif- ference between, for example, the annual rate of change in the index from Monday 3 January 2000 to Wednesday 1 January 2001 and the corresponding annual rate of change between the complete months of January 2000 and January 2001.

This will not be the case if inﬂation is rapid or the rate changes signiﬁcantly during the year.
The diﬀerence between 1 January and 1 February and average January to average February inﬂation rates may be diﬀerent – particularly if so-called ‘‘sale’’ periods are 81 CONSUMER PRICE INDEX MANUAL: THEORY AND PRACTICE limited by laws or local ordinance, as they are in some countries.

For certain products with high index weights, where price changes are sudden and tend to aﬀect the whole market on about the same day, the choice between time period and point is important.
Examples are petrol, electricity and telecommunication prices.

Here the case for an average price for the period is strong.

Obviously, the weights should relate to the periodicity of the col- lection taking into account the appropriate expenditure and pricing periods (for example, if prices are increased one-third of the way through the period then two-thirds of the weight should reﬂect the higher pricing).
6.10 Not all price observations can be made within a single day, let alone at one point in time during that day.

This is particularly true of local price collections, but may also be true of central price collections, depending on the resources available at head oﬃce.
In practice, the real issue is whether the observations are spread over a few days to provide an approximation to a point-in-time estimate (for example, Monday to Wednesday to repre- sent prices on a speciﬁc Tuesday) or spread over the whole month to provide an estimate for the average for that month.

6.11 It should also be borne in mind that the sam- pling variance will diﬀer according to whether a period or point-in-time index is compiled and, in the case of the latter, the frequency of collection.
In considering the tim- ing and frequency of price collection, consideration also needs to be given more generally to the trade-oﬀ between statistical accuracy and cost.

It should be noted that collecting prices locally from shops is normally a rela- tively expensive activity.

In practice, the budget for price collection usually limits the available options.

6.12 The desired frequency of price collection may vary by commodity, depending on how frequently the prices to be observed change.
For example, it is possible that prices charged by public utilities, central or local government fees or charges, or mail order catalogue prices will change annually or quarterly according to a known timetable, and therefore these price collections can be carried out according to the timetable of changes rather than every month.
In contrast, food prices – where shopkeepers may review the prices they charge on a continuous basis to reﬂect market conditions and the prices charged by their suppliers – need to be collected more frequently.

Clearly, statisticians will need to be absolutely sure about the frequency of price changes for any speciﬁc good or service before coming to a decision to collect prices on a less frequent basis.
They will also need to keep in touch with current pricing policies to monitor whether the position changes, so that they can immediately reﬂect these changes in their price collection procedures.
In addition, statisticians need to be aware of any unusual price changes that could be missed by less frequent price collections, for example, changes to indirect taxation rates or one-oﬀ timing changes of increases (for example, service providers moving annual increases from April to March or prices of school dinners changing each term, with terms starting in diﬀerent months from one year to the next).

6.13 Another point to note is the timing of pub- lication of the resulting price indices.
There may be legal 82 constraints on the timing of the publication of indices.

In such cases, prices must be collected in time to allow quality assurance, processing and aggregation pro- cedures to be completed before the deadline.

6.14 As mentioned above, in cases where inﬂation is stable and where collection costs are not an issue, col- lection can be spread over a whole month.
In these cases, diﬀerent neighbourhoods should be scheduled for price collection at diﬀerent times of the month according to a regular pattern to be repeated every month.

This not only makes the use of the collectors’ time more eﬃcient, but also has the advantage of providing a spread of collection dates for many representative items.
It is also important that individual price observations are carried out at the same time each month so that the index does not move as a result of a change in the length of interval between collection dates.
A further important con- sideration, particularly of consequence in Middle East- ern countries, is in cases where prices can vary by day of the week (for example, depending when market day is) or time of the day according to various marketing special oﬀers to attract more customers at less busy times or to reﬂect the freshness of the goods.

6.15 When the aim is to compute a point-in-time index, price observations need to be spread over a very small number of days each month.
The interval between price observations should be uniform for each outlet.

Since the length of the month varies, this uniformity has to be deﬁned carefully.

6.16 Preferably, days of the week and times of the month should be chosen taking into account when purchases are concentrated and where prices and goods in stock are known to be representative of the month as a whole.
In Middle Eastern countries, results of the household expenditure surveys suggest that most house- holds do the shopping on the day of the souk (market).

It should also be borne in mind, however, that retailers may be less prepared to cooperate when they are busy, so a balance needs to be struck between the ideal timing for collection and the impact on response rates.
It should be noted that a ﬁxed interval is impossible because of the varying length of a month and the timing of public and religious holidays.

One solution is to take sequences of four and ﬁve weeks, so maintaining a relatively stable monthly or quarterly period; another is to follow a rule such as collecting on the regular market day or on Wednesday through to Friday of the ﬁrst full week in the month.

6.17 Price collection days (and sometimes times) need to be set in advance.
In some countries or econo- mies, decisions need to be taken in advance about whe- ther and how these days should be kept conﬁdential to avoid key sources, such as major stores or governments, adjusting prices for collection days and thus distorting the price indices.

It is nevertheless important for public perception about the integrity of the index that a statis- tical oﬃce is able to explain the procedures used for setting collection dates and the underlying objectivity of its method.
Any price collection agencies carrying out the collection for national statistical institutes need to know collection dates a long way in advance for resource planning purposes.

In addition, any data suppliers who supply prices direct to head oﬃce staﬀ need to know the collection date a short time in advance to be able to prepare and supply the necessary price returns.

6.18 Regular timing is particularly important when inﬂation is rapid.
Where there is a speciﬁc collection day, it is most important that the most volatile prices are collected on that speciﬁc day rather than the days around it.

Particular items may include fresh fruit and vege- tables, fresh meat, and items subject to varying indirect taxation and duties (such as tobacco and petrol).

In the case of foodstuﬀs sold in marketplaces, the time of day as well as the day of the week is important.
In Middle Eastern countries, at least, these prices are usually higher in the mornings and lower in the evenings.

6.19 Price collection days need to be set after con- sidering a variety of factors that aﬀect prices and shopping patterns.

Holidays and weekends should be avoided except for items with large sales during these times, such as petrol, leisure services and entertainment (for example, restaurant meals and tourist attractions).
Some countries have limited shop openings on some days or half-day closing on other days, which can limit the number of prices that can be collected or bias the location sample to certain types of outlets or service providers.

On the days approaching long holiday peri- ods when many shops are shut, there can be limited supplies of fresh food and many abnormal price reduc- tions to clear stocks before the shops shut for the holiday period.
The implications of any sale periods controlled by law should also be considered.

6.20 Whether collection is continuous or point-in- time, the interval between successive price observations at each outlet must be held constant by visiting that outlet during a ﬁxed time period each month (or quarter).
6.21 Another issue is raised by the pricing of tariﬀs (telephone charges, for example, depend on the time of day and the destination of the call), variable pricing policies dependent on demand (sporting and leisure fees, for example, depend on the time of day – peak demand times attracting higher prices) and prices where there is potentially limited availability (such as air, rail and taxi fares).
For each of these, price collections should be made consistently over time and in a way that represents consumer purchasing patterns.

The selection of the representative items should represent consumer behav- iour (for example, air fares may be priced 6, 3, 2 and 1 month in advance and include last-minute booking options too) and be weighted by consumer spending patterns (for example, weighting together prices for peak and oﬀ-peak entry to a swimming pool).
6.22 A ﬁnal point to note is that with the point-in- time approach, major price setters, notably the govern- ment, can inﬂuence the index according to whether their price changes take eﬀect on a day just before or after the day for which their price information is obtained, or on the day of collection.
Since prices are often collected centrally from such price setters, it should be possible to obtain information from them about both the amount and timing of price changes at the end of each month, so that in applying the period-of-time approach, an aver- age price for the whole month can be calculated.
For example, if electricity charges are made quarterly and PRICE COLLECTION prices increase part way through the 3-month period, individual customers’ payments could include 0, 1, 2 or 3 months at the higher rate.

Taking account of hyperinﬂation 6.23 Special arrangements may need to be put into place where there is hyperinﬂation.
In these circum- stances it becomes even more important that the prices of individual items in individual shops are collected at precisely the same time each month, otherwise mislead- ing ﬁgures may result.

Consideration should be given to the more frequent collection of prices and correspond- ingly a more frequent compilation of the index.

Where prices are normally collected on a quarterly basis, it may be sensible to collect them more frequently.
If this is not feasible, it may be appropriate to up-rate prices proportionately by some relevant indicator to provide an approximation to a monthly index.

If this is done, however, great care needs to be taken in choosing the appropriate comparator, particularly as relativities between prices can change dramatically in periods of hyperinﬂation.

6.24 In some circumstances, rapid or frequent price changes may be associated with certain items only and action should be taken accordingly.
By way of example, food prices may rise disproportionately because of a bad harvest and it may be sensible to increase the frequency of the index for food items only.

Alternatively, a simpler way of dealing with this situation may be to monitor a small number of relevant prices on a regular basis without producing a full price index.

Such sub-indices could be published separately or used to up-rate the later prices collected in the period, as mentioned above.
These items may be chosen according to their importance for the family budget and whether they are particularly susceptible to big price increases.

Item speciﬁcation 6.25 Speciﬁc representative items should be chosen to be typical of price movements in the consumer price index basket.

An item consumed by households or individuals that has a price is a deﬁnable good or ser- vice.
In some cases, however, such as a` la carte restau- rant meals, cars (where the purchaser may be able to buy optional extras on top of the basic model) and car ren- tals (where insurance may be extra), a decision needs to be taken about whether to treat the package as a single item or whether to price components separately.
As a general rule, the package should be considered a single item when it can be expected that the oﬀer is not tem- porary and where the purchaser typically buys the whole bundle of goods and services on oﬀer.

Otherwise, the components should be treated as separate items and individual cost quotes obtained.

Where the purchase is not normally of the whole bundle, it is usual to be able to pick up individual quotes for the diﬀerent parts of the package.
This provides some indication of whether packages or individual items are being purchased.

6.26 Ideally, the selection of items should be based upon a complete census of relevant transactions relating 83 CONSUMER PRICE INDEX MANUAL: THEORY AND PRACTICE to diﬀerent items purchased by individuals.

In practice, this is not normally universally available, although in some countries useful data may be generated by point- of-sale and scanner data.
6.27 How tightly deﬁned (narrow) or general (broad) the item speciﬁcation should be is an issue of great theoretical and practical importance.

Whether speciﬁ- cations are broad or narrow can vary according to individual circumstances and may diﬀer throughout the basket of goods and services being priced.
Narrow item descriptions are generally more eﬀective for control- ling sample representativeness (assuming that a reliable sampling frame or set of reference data is available) and for controlling for quality diﬀerences, and can also reduce the variance of prices and price relatives, thus optimizing the performance of some aggregation for- mulae.

But they can result in a smaller achieved sample, as there is less ﬂexibility for price collectors to choose an appropriate item in a particular shop.
In contrast, broad item descriptions can increase the size of the achieved sample but can be more diﬃcult to control for sample representativity and will normally result in higher variances.

6.28 In some countries, prices for clothing are speci- ﬁed very tightly to ensure that quality diﬀerences are minimized.

A description might be as detailed as ‘‘knit- ted top; mid-season; with sleeves; no collar; no buttons; made in Morocco; acrylic; mid to light thickness’’.
In comparison, a general description used in another con- sumer price index for an equivalent item might be ‘‘men’s formal shirt; long sleeved’’.

6.29 Whichever of the two approaches is used, rules should be established for selecting representative items that ﬁt the item descriptions (for example, best-selling lines as reported by the individual retailer, or items selected by probability proportional to size sampling).
It is important that the representative items chosen, whether with tight or general product descriptions, are actually representative of the consumer spending pat- terns.

There is no point, for instance, in pricing an item that is rarely sold but looks good in the shop window, or is just in a convenient location for the collector to ﬁnd it each month.

The rules for selection should also take account of the sampling methodology underlying the selection of shops.
There is a stronger argument for using some form of probability sampling for the selec- tion of items using tight descriptions when the selection of shops is more loosely speciﬁed, and vice versa.

This is because the broader the item descriptions and the more loosely controlled the item selection in the ﬁeld, the more the representativity of the sample is reliant on the quality of the initial selection of shops.
6.30 It is also important under either speciﬁcation regime that instructions to price collectors give an ade- quate description of the item to be priced.

For instance, in the case of a washing machine the information required for a tight speciﬁcation may include make, model number, capacity, whether automatic, whether top or front loaded, and spin speed.
As well as providing eﬀective sampling control, this will also be useful infor- mation if a price collector has to choose the nearest equivalent should the particular model cease to be 84 available.

It is important that the number of prices obtained for tightly speciﬁed goods or services are reg- ularly reviewed so that speciﬁcations can be updated if these items are being phased out or if consumer pur- chasing patterns are changing.
6.31 A loose speciﬁcation may simply specify a washing machine with a particular range either for the capacity or the spin speed.

In this case it is still impor- tant that the collector records a detailed description of the washing machine being priced to enable the selection of a comparable model if that model is discontinued or so that a future collector can carry out the price col- lection when the original collector is not available.
Collection procedures 6.32 An important consideration in the collection of prices is the scope of the price index being constructed.

For example, should black market or contraband goods be priced as part of the price collection?

In general, if such purchases constitute a signiﬁcant part of expendi- ture then there is an argument in principle that they should be considered for inclusion.
This, however, leads to price collection diﬃculties such as ﬁnding the neces- sary outlets, which may be transient and not advertise themselves, as well as the actual pricing of goods and services.

Another diﬃculty regarding scope concerns activities that are considered illegal in some countries but not in others (for example, prostitution, gambling or sales of alcohol).

6.33 The greatest diﬃculty in collecting prices arises for goods and services in economies where bartering plays an important part.
Examples range from prices for cars, which can be individually negotiated (including the possibility of trading in an old car), to market stalls in some communities.

Ultimately the price obtained depends on the likelihood of a real purchase, and the negotiating skills of the price collector, as well as factors such as how desperate the retailer is for a sale.

Ideally the price col- lector should obtain the price that a consumer would actually pay.
In some cases it may be appropriate to look at alternative price collection methods or indicators instead (such as the advertised price, which could be assumed to move in the same way as the bartered price depending on circumstances).

6.34 In some Middle Eastern countries where prices vary according to the time of day and where prices are not usually advertised (for example, in the souk), it is necessary to employ a variety of collection procedures.
Prices for fresh meat and vegetables may be collected including a morning, three to six times in a day, lunchtime and evening visit.

In addition, collectors can be trained to recognize ‘‘deceptive’’ prices and can be encouraged to linger and listen to transaction prices for genuine sales.

6.35 Diﬀerent collection procedures may be applic- able for diﬀerent outlets.
Permanent outlets can some- times be selected on the basis of a sampling frame either held centrally or through local enumeration (see Chapter 5).

In the souk or marketplace it may be appropriate to use other collection procedures, particularly where the opening times and the variety of stalls and goods for sale vary at diﬀerent times.
In these cases, the item list may be restricted to items known to be available in the souk and price collectors may be asked to obtain a ﬁxed number of price quotes for each item, the number being determined by local knowledge of the varieties on sale and the var- iation in price.
Some items, such as fruit and vegetables, may warrant more quotes than others and prices may need to be collected at intervals throughout the day (for example, three times in the morning, three times during lunchtime and three times in the afternoon or evening) to ensure that any variability in price according to time of day is taken into account.
Consideration may also need to be given to collecting from farmers (who travel to the souk to sell their wares) and middlemen (who buy the food from the farmers and then sell it on).

6.36 Another diﬀerence between countries may be where a signiﬁcant proportion of expenditure occurs abroad and then the purchased items are imported by individuals (for example, car sale markets in Lithuania are frequented by the population of other Baltic States).
Under such circumstances, price collections need to be considered in terms of the scope of the index (for example, should prices in other countries be considered?)

as well as the more complicated matter of pricing the same or similar quality car each month.

6.37 An overview of local price collection for straightforward outlets is given in Figure 6.1.
This dia- gram assumes that outlets have already been enumerated and selected, that the shopkeeper or head oﬃce of a chain has agreed that price collectors may visit on a regular basis, and that usual identiﬁcation formalities on arrival and departure will have been carried out.

In addition, it assumes that item selections have already been made in previous months.
This is usually best done on a separate pre-collection visit where the price collec- tors introduce themselves, familiarize themselves with the shop, and explain the price collection procedure to the shopkeeper.

6.38 The diagram details diﬀerent decisions and actions that a price collector must make to price any individual item.

The diagram starts with arrival of the price collector at the outlet, at a mutually agreed time, which may or may not coincide with the usual shop opening times.
Having gained entry to the outlet (or a replacement outlet), the collector attempts to price the necessary item or items.

In a straightforward situation the item is immediately available for purchase and priced.

More complicated situations arise when the item is diﬀerent from the previous collection in some way (such as in size, description, weight, or quantity), in which case the usual procedure is to price the item and report the facts to head oﬃce.
Finally, if the item is unavailable, another item has to be selected as a repla- cement comparable item or a replacement new item.

Having priced all the items required in that outlet, the collector can move on to the next outlet.

6.39 The choice of a comparable item is made using the same bundle of key characteristics as that potentially aﬀecting the price.

For example, the brand name, wash cycles, capacity, energy consumption and spin speed may aﬀect the price of a washing machine.
PRICE COLLECTION 6.40 The most complicated situation arises when a diﬀerent item that is not of a comparable quality has to be priced.

How this is treated depends on the procedures in place for adjusting prices for changes in quality.

For example, quality changes may be treated implicitly by considering the item as a new item with an imputed base price.
The latter may be calculated by head oﬃce staﬀ who may require supplementary information from the price collection or by the price collector in the outlet with the assistance of sales staﬀ.

6.41 Seasonal items require special attention.

In some situations, seasonal items such as fruit, vegetables or clothing may not be available for pricing all year round.
One way of reﬂecting this in the index is to use seasonal weights, which diﬀer for each month of the year and reﬂect expenditure information from house- hold budget surveys or other sources.

Alternatively, other seasonal items may be priced at diﬀerent times of the year to directly replace the unavailable items (for example, bathing costumes and shorts may be priced for six months, and gloves and scarves for six months).
6.42 One possibility for data collection is to collect some items less often than monthly, thereby making possible a larger total sample.

Many items in the United States Consumer Price Index (CPI) are collected only bimonthly in any given area; similarly, the rent samples are divided into six panels, each priced twice each year.

It makes the calculation more complex but may be more eﬃcient from both a statistical standpoint and also for collectors.
Price collection techniques 6.43 For many items, prices will be collected locally by price collection agencies employed by the national statistical institute, or their own employees, visiting retail outlets and recording current prices for an agreed selec- tion of items.

But some prices may be collected centrally from catalogues, by retailers providing list prices cover- ing a range of outlets, by telephone, fax, letter, emails or from Internet sites.
All these methods may be cost- eﬀective or necessary to represent diﬀerent aspects of consumer purchasing behaviour and so it is not sur- prising that many statistical oﬃces use a variety of data collection techniques.

In addition, such price collections can allow for the implementation of speciﬁc methodo- logical procedures by head oﬃce staﬀ (for example, quality changes).

Either local collection agents or head oﬃce staﬀ can use these varying collection methods.
Examples of price collection techniques include the fol- lowing: Prices may be obtained from mail catalogues to represent a certain type of retail outlet, or where high street catalogue stores have nationwide coverage with uniform pricing policies.

Increasingly in some coun- tries mail order suppliers are oﬀering their own Inter- net services.

In the case of both mail order and Internet shopping, care has to be taken to treat delivery prices and sales taxes consistently and correctly.
Prices may be obtained over the Internet either for convenience (where major stores oﬀer the same prices 85 CONSUMER PRICE INDEX MANUAL: THEORY AND PRACTICE Figure 6.1 Price collection procedures Is the outlet open?

No Is it permanently shut?

Yes No Yes Yes Is the outlet willing to cooperate?

Is refusal because authorized person is not available?

No Try again later if the instructions say so, otherwise report the facts.

Yes Is the product type to be priced currently available to purchase?
No Report the non-availability and the reason for it.

Yes Yes No If the variety is neither seasonal nor expected to be available to purchase again next month, then: Yes Is there a large difference from the price recorded last month?

Is this because offinal clearance for damaged or soiled items?

Disregard the price.

No No Report the facts to HQ Price next item.


Report the non- availability and the reason for it, providing the full description of the unavailable variety if central office does not already have it.

Select another outlet according to instructions.

Report it.

Yes Is the collector authorized to select a replacement outlet?

No State the reason, e.g.

sale, special offer, black market price, replacement outlet, replacement item.

Is the collector authorized to find a replacement?


No Is the replacement the same kind as its predecessor?

No Yes Yes Select another variety likely to remain available.

Record a description of it in sufficient detail both to cover quality differences and to enable exact identification.

Are both prices available for the same month?


No Estimate and report the amount of price difference reflecting the value of the quality differences revealed by the descriptions of the original variety and its replacement.

Yes Yes Is the collector authorized to make quality judgements?

No Report the facts to HQ.

Record the price and, if it is non-standard, also record the weight, size or quantity.

No Is the outlet likely to be permanently shut next month?
Yes Is the variety expected to become permanently unavailable next month?

No Price next item.

Yes 86 on the Internet as in the shops) or through necessity in order to maintain a representative sample where this type of retail outlet is increasingly used (for example, for books).

 Some retailers have national pricing policies with no individual pricing discretion, even for sales and special oﬀers.
In these cases, a single store can be visited or the retailer’s head oﬃce may agree to supply a single price list (covering all items or prices for the speciﬁc selected items).

 Prices may be obtained over the telephone or by fax where there will be no ambiguity in price because the item being priced is standard and the contractor will quote a standard charge (for example, electricians may be telephoned for charges for providing a new single electricity socket).
In addition, obtaining a price over the telephone will reﬂect what the consumer will often do in practice.

A further factor is that many service providers (such as plumbers or window clean- ers) tend not to work from retail outlets and it would be diﬃcult to visit due to their variable working hours oﬀ-site at the customer’s own premises.
Prices may be obtained by letter, fax or email, accompanied by relevant head oﬃce forms for com- pletion and return in cases where central oﬃce col- lection is deemed more eﬃcient or where local price collection is not possible (for example, tariﬀ prices).

Examples include prices collected from a sample of local authorities, insurance companies, public utilities, and telephone companies.
Prices may be obtained from other government agencies or regulatory authorities, which can act as intermediaries in the price collection process.

In some countries, for example, this would be the case for electricity prices.

 In some cases, secondary sources can provide data on speciﬁc goods.

Two examples, taken from the United States CPI but by no means unique to that country, are airline fares and used vehicles.
A sample of scheduled airline ﬂights is selected using detailed ticket data from the United States Department of Transportation.

Monthly pricing is then carried out by online reference to a private-sector computerized fare database widely used by travel agents and others.

In the case of used cars and trucks, both sampling and pricing employ published data from a dealer trade association.
The beneﬁts of using secondary data may include larger sample sizes, faster or less expensive access to data, or the avoidance of particularly diﬃ- cult collection problems.

6.44 When using other sources such as catalogues or the Internet for prices, special care must be taken to ensure that they are correctly recorded with or without sales taxes, or with or without delivery charges.

In these cases, procedures should include a check that the prices are relevant for the index period.
6.45 It is important to remember that all the usual price collection principles and quality assurance con- cerns remain relevant for prices collected from the Internet (including the need for detailed descriptions, immediate availability of the item for purchase, treat- PRICE COLLECTION ment of special oﬀers, and possibility of substituting comparable or new items).
6.46 Where prices are taken over the telephone it is recommended that the retailer be visited occasionally, where practicable, to maintain personal contact and response rates and to ensure that no misunderstandings of items or pricing are occurring.

As far as possible, prices collected by telephone should also be conﬁrmed in writing to provide conﬁrmation for quality assurance procedures (see Chapter 12).
6.47 Many households may be unable to access the Internet, and Internet shopping provides additional services, such as home delivery.

This means that col- lecting prices from the Internet can be considered either as the introduction of a new outlet or a new item.

In both instances action should be taken as part of the procedure for maintaining a representative sample at the time of the regular updating of the item and location selections, usually at the time of chain linking.
It should be noted that consideration will also need to be given to whether a move to Internet shopping involves a quality change.

For example, in the case of food shopping, free delivery may be included for payment over a certain level, or the average ‘‘use by’’ date may diﬀer from that found in traditional outlets.

6.48 The scope for improving the eﬃciency of data collection can increase with the arrival of technological advances in the marketplace.
New collection methods are continually becoming available, particularly in the technologically advanced countries.

Methods for future collection include touch-tone dialling facilities and scan- ner data, both of which have the advantage of oﬀering new ways for businesses to reduce the burden or incon- venience of supplying data.

6.49 It should be remembered that to keep the index representative, it might be appropriate to collect prices for an item in more than one way.
For example, peo- ple may buy books from catalogues, from a variety of shops (bookshops, newsagents, supermarkets, depart- ment stores, and so on) and through the Internet.

In these circumstances, it is appropriate to collect prices through all types of outlet where transactions are sig- niﬁcant.

Questionnaire design 6.50 Good design of the questionnaire form (or its electronic equivalent) is essential for the successful col- lection of prices.
Not only is it important that the price collectors ﬁnd it easy to use, but the format and layout should facilitate the extraction of data (price, item description, comments, and so on) by head oﬃce for eﬀective quality assurance.

6.51 The ﬁrst step in designing a questionnaire is to deﬁne the information that needs to be gathered and how it will be collected.
Diﬀerent forms will be appro- priate for each of the collection methods that are deployed, for instance visiting retailers compared with collecting by post.

There will, however, be a number of common principles.

The questionnaire should be prac- tical for the price collector to use in the ﬁeld and should also facilitate basic quality assurance.
It is for the latter 87 CONSUMER PRICE INDEX MANUAL: THEORY AND PRACTICE reason that it has been argued that the price recorded last time the item was surveyed should be shown on the questionnaire, as this will prompt the collector to ask questions if the previous price is very diﬀerent from the current price.
The converse to this argument is that the recording of the last price may mistakenly lead price collectors to identify the item to be priced by reference to price rather than item description, or at the extreme to estimate the price or repeat the previous price without actually visiting the shop.

6.52 It should be remembered that, at the time an index is chained, the questionnaire will need to list all items included in the old as well as the new basket.
For example, an index chained annually on January prices requires both the old sample of locations and items and the new sample of locations and items for the base month.

6.53 An example of a price collection form is given in Appendix 6.1.

This example is of a form used by the collector for recording prices when visiting an outlet and could be either a paper or electronic version.
It is also possible to ask the shopkeepers concerned to complete the form themselves and to send it to the national sta- tistical institute.

Such a form may therefore serve for reporting as well as for collecting.

If the form has space for recording prices over a whole sequence of months, the collector may keep the form and transcribe the prices from it each month onto a separate report form, which is sent to the national statistical institute.
Where the form used for collection is also used for reporting, there are two main possibilities: either the form has space for recording prices over a whole sequence of months, and the form is shuttled backwards and forwards monthly between the collector and the oﬃce; or new forms for collection and reporting are printed out by the computer each month.
In the latter case, if considered desirable, the form may contain the prices recorded in the previous month alongside the spaces for recording the current month’s prices.

It should be noted that the transfer of the prices to another form or system, whether done by computer or manually, may lead to transcription errors.
Increasingly, the use of an electronic version of the questionnaire on a hand-held computer or ‘‘personal assistant’’, with built-in validation checks, is seen as advantageous for local price collection by price collec- tors.

The data can then be transferred electronically from collector to head oﬃce via a variety of intermediate steps for further validation checks by the price collection agency.
6.55 It is recommended that price collectors be required to provide full descriptions of the items being priced.

This enables checks to be put in place to ensure that collectors are properly following instructions, par- ticularly on the selection of items to be priced.

It also ensures that any changes, including changes in the quality of the items, are being properly identiﬁed, with enough detailed information to enable decisions to be taken on quality adjustment.
Price collectors should be given a checklist or set of codes to record relevant information on changes relating to outlets, items or prices.

The information needs to be systematically col- lected.

For instance, codes to help with quality adjust- 6.54 88 ment need to reﬂect those characteristics that most inﬂuence price.

Prior research, for example, based on the hedonic method, can help to predetermine these (see Chapters 7 and 21).
6.56 Codes for managing the sample of outlets may include: – closed down: outlet permanently shut or closed down; – temporarily unavailable: outlet temporarily closed, but likely to be open next month; – refusal: owner or staﬀ refuses to cooperate; – change of details: change of ownership or name, or change of purpose.

6.57 Continuity is one of the most important prin- ciples of price collection.
As the index measures price changes, it is vital that the same item is priced every month in order to establish a true picture of price changes.

So if, for example, a jar of a supermarket’s own brand strawberry jam has been selected, that particular brand and ﬂavour should continue to be collected; if it is out of stock, another brand and ﬂavour should not be used without further investigation to establish whether this is a temporary situation or likely to be permanent.
In the latter case, and if another ﬂavour of the same brand, size and quality is available, then this item should in normal circumstances be chosen as a ‘‘comparable’’ item and the item description suitably amended.

If a diﬀerent brand, size or quality product is available then this should be selected as a ‘‘new’’ item, but only where no comparable items are available.

The same principles apply to other items, such as clothes, and fresh fruit and vegetables.
With clothes, it may be important that col- our, fabric, country of origin, logos and size are speciﬁed to ensure that the same item is priced each month.

For fresh fruit and vegetables, useful attributes to record may be ‘‘country of origin’’, ‘‘class’’ and variety.

For electrical equipment, it may be the speciﬁcations and features given in the manufacturer’s catalogue that are important.
6.58 It is not possible to be prescriptive because the concept of equivalence will vary between diﬀerent coun- tries; but for practical purposes it is important that a detailed description of the items being priced is recorded.

Item descriptions will assist the price collector and head oﬃce in choosing or conﬁrming the suitability of a replacement for an item that has been withdrawn and will also help identify changes in quality.

The focus should be on recording price-determining characteristics.
6.59 Should the regular price collector, for whatever reason, be unable to carry out the normal collection, full and accurate descriptions will enable a relief collector to carry out the collection without any doubt as to the correct items.

6.60 Most of the time, the item will be exactly as collected the previous month and all that will be recorded is a new price.
However, should there be a change or uncertainty in the item, then it will be necessary for price collectors to use their own judgement and to inform head oﬃce, bearing in mind that head oﬃce staﬀ are responsible for making the ﬁnal decision.

A pre-coded speciﬁcation will be less time-consuming and will pro- vide better guidance to the price collector on what information should be reported.
The codes might include: Comparable (C): The original item is no longer stocked but a similar alternative has been collected that does not diﬀer in terms of major attributes.

The price is likely to be in a similar range although this may not always be the case.

New (N): The item has been replaced by something new that is not really comparable but is equally representa- tive of that commodity group.
If possible the collector should try to ﬁnd out the price of the ‘‘new’’ item in the chain link or base period.

Sale or special oﬀer (S): A price decrease because of a genuine sale or special oﬀer, with a sale or discount sticker present.

This does not include damaged or out of date stock or clearance goods.

The latter should never be included.

A price reduction where there is no notice of a sale or special oﬀer is not a ‘‘sale’’; the item should still be priced, but without the S indicator code.
Recovery (R): A return to the normal selling price, for example after a sale or special oﬀer.

This need not be a return to the same price as before the sale or special oﬀer.

Temporarily out of stock (T): Guidance will need to be given to the price collector concerning the meaning of ‘‘temporarily’’ (in terms of expected duration, which may vary for diﬀerent items).
It may be advisable to replace items immediately (for example, fashion cloth- ing, if it is unlikely that the identical item will come back into stock).

Typically, T indicators should not be used for more than two consecutive months – in the third month, a replacement should be chosen.

In food outlets, it is very unusual for items to go permanently out of stock.

The collector should always try to check future availability with the retailer.
Missing (M): Used where the outlet has never stocked or no longer intends to stock an item and there is no appropriate alternative item.

In these circumstances it is recommended that the item is checked at subsequent collections to ensure that a suitable replacement item has not come into stock.

Weight (W): A permanent weight or quantity change to the product.
Query (Q): Such a code may be used to supply extra retail information to head oﬃce (for example, ‘‘10 per cent extra free’’, ‘‘3 for the price of 2’’, or a strange price diﬀerence that is not covered by one of the other indi- cators, such as a bumper issue of a magazine at an increased price).

Arrangements need to be in place for head oﬃce to respond to these comments and to treat the price quotes accordingly.

6.61 The use of these codes is illustrated in Appen- dix 6.1.
Even if the retailer says there have been no price changes since the previous month, the price collector should conﬁrm prices anyway.

This will require some diplomacy, but it is important because it is easy for the shopkeeper to overlook a small number of price increases, forget when the last increase occurred or even deliberately mislead the price collectors.

The use of codes is important for operational reasons.
For exam- PRICE COLLECTION ple, if an item is unlikely to remain available the next month, then a substitute can be selected in advance and an overlap price collected.

6.62 As a general rule, a price should be recorded only if the exact product being priced is on display and immediately available for sale.

No price should be recorded if a product is temporarily out of stock.
For certain large items such as furniture, however, where the item must normally be ordered, the price should be recorded as long as the retailer conﬁrms that it is avail- able for delivery within an ‘‘acceptable’’ time period.

6.63 Some food items, such as meat, ﬁsh and cheese, can be sold in variable weights, so it is sensible to collect prices per unit of weight.

This should be taken from the package labelling or calculated directly by the collector.
Roughly the same package size and type should be used each month, as the unit price might be lower for larger pack sizes or diﬀer between package types.

Other items, such as eggs, are often sold in speciﬁed quantities.

For these, it is essential that collectors record prices for the speciﬁed quantity, as total and unit prices usually depend on the number bought.
If X eggs are to be priced and the price for the number is not quoted directly, then the price of one egg can be obtained and multiplied by X to get the required price.

Care does need to be taken, however, to ensure that unit price does not decrease with quantity.

Another example is mint.

This herb is often sold in bunches of variable size, so a number of bunches should be weighed and priced to obtain a price per kilogram.
6.64 Certain food items, such as fruit or vegetables, are more diﬃcult to price as some outlets might price items per number purchased while others might price by weight.

For example, peppers may be priced by weight or by unit no matter what the size.

Garlic may be priced per bulb, clove or by weight.

Various types of berries may be priced by weight or by punnet, which may diﬀer in size or how full they are.

In these instances, care has to be taken with the product descriptions.
Collectors need to be aware of the importance of collecting the same thing from one month to another, so that genuine price changes are recorded and not quantity or quality price changes.

6.65 The use of hand-held computers for local price collection provides more scope for quality assurance both in the ﬁeld and at head oﬃce, without some of the disadvantages associated with paper forms.

Price col- lection using hand-held computers is discussed in more detail below.
Use of electronic forms on ﬂoppy disks or by email, for example for central price collection from the head oﬃces of large retail chains, may be more cost- eﬀective than sending price collectors into individual outlets.

But in these circumstances, care needs to be taken to check that there are no price variations between diﬀerent outlets in a chain and that any special oﬀers given locally are covered.
Where there are such local factors, adequate account will need to be taken of them, otherwise the price recorded for the index may be mis- leading.

6.66 A decision needs to be taken over whether large retail chains should be placed in separate strata (treating the chain rather than an individual outlet as the 89 CONSUMER PRICE INDEX MANUAL: THEORY AND PRACTICE sampling unit) or whether a sample of outlets from each chain is taken (thus taking as the sampling unit an outlet from a particular chain).
As a general rule, a retail chain with no national pricing policy cannot be treated as a single sampling unit, but it may be possible to visit only a few of their outlets if it can be established that each outlet visited reﬂects the chain’s prices over a wide area.

In these situations, it is usual to approach the manage- ment of the chain’s head oﬃce to conﬁrm their pricing policies and obtain permission for the collection.
Each year, when approached again for permission to continue the price collection, the management should be asked to conﬁrm that their regional pricing policy remains unchanged.

The prices collected are then given a weight to reﬂect the market share they represent, in the same fashion as the weights applied to prices collected cen- trally for a chain where there are no price variations between outlets.
Issues relating to central collection of prices from local businesses and outlets (for instance, over the telephone by staﬀ located in the national sta- tistical institute) and the collection of prices for retail chains from their corresponding head oﬃce are dis- cussed further below.

Field procedures 6.67 Adequate ﬁeld procedures are required to ensure that the quality of the price index is not com- promised through errors in price collection.
Price col- lection needs to be carefully planned and managed, and eﬀective instructions and training given to price collec- tors.

Most prices are likely to be collected through price collectors visiting individual outlets.

Guidance on the organization and management of ﬁeld procedures relating to local price collection is given in Chapter 12.

In some circumstances it may be more eﬃcient for prices to be obtained from one source rather than through surveys in the ﬁeld.
These are covered in the section that follows.

6.68 Central and head ofﬁce collection 6.69 One form of central and head oﬃce price col- lection is where price data representing a number of shops are collected from one single source.

This can take place when chains of shops have proven national pricing policies, with no local variations between stores in terms of either price normally paid or special oﬀers and dis- counts.
In these cases, the chains’ outlets should be excluded from local price collection and the prices col- lected should be weighted according to the market share of sales.
6.70 The selection of this type of central collection and calculation is usually dependent on one or more of the following considerations: national or local pricing policies; available sources of data (including willingness of chains to assist in this way and forward commitment by them to provide data centrally); data presentation and format (advertised prices or average transaction prices provided by email, on ﬂoppy disk or paper); reference point of available data (price lists match the collection day or period); and frequency of price changes.
90 6.71 Central price collection may also be appro- priate for some service prices.
These could include: – fees set by a professional or trade association or union; – charges for public utilities or services provided by deregulated (and regulated) bodies or government (such as: water, gas and electricity tariﬀs; bus and train fares; birth, marriage and death registration fees); – prices centrally determined by government (for example, fees to be paid for services, such as health care and education, that may be partially or fully funded by government); – taxes and licence fees paid to government (for exam- ple, television licences and vehicle excise duties).
In some instances, data may need to be requested from regional authorities, for example where there are region- al utility providers.

6.72 Data may be requested in writing, or by tele- phone or electronically.

Where letters are sent, con- sideration should be given to using oﬃce automation for the generation of data requests (for example, mail merge facilities), logging responses, monitoring progress and sending reminders to non-respondents.
Useful categories for informing progress might include: return received; return being checked; query sent and awaiting resolution of query; ﬁgures ﬁnalized.

6.73 The greatest gains from electronic reporting of centrally collected prices are likely to be eﬃciency gains from automation, better work monitoring and fewer problems arising from transcription errors.
The risk – one that is associated with all central price collection – is that the impact of an undetected error can be com- pounded because of the relatively large weight that may be placed on one price or set of prices.

Clearly, this factor should be reﬂected in the quality assurance pro- cedures as well as in the sampling procedures.

It has been observed that national statistical institutes can be slow in reviewing their quality assurance procedures following a move to greater central price collecting.
This can lead to a disproportionate amount of eﬀort at head oﬃce being focused on the checking of local prices.

This is particularly so if local prices have been rigorously scrutinized in the ﬁeld; any individual error will not have a noticeable impact on the index unless it is part of a systematic bias, for example arising from inadequate instructions to collectors.
6.74 Providers of goods and services may send either a full price list or a tariﬀ from which an appropriate sample of prices and weights can be extracted or just those prices required for compilation of the index.

In some instances, for example a regional transport author- ity, it may be acceptable for data to be provided in the form of a price index.
In these cases it is clearly impor- tant to ensure that the index has been calculated accu- rately and in accordance with the requirements of the consumer price index, using agreed methodology, and that the central oﬃce exercises strict quality control.

The latter may be done by, for example, checking the com- putation once a year or more frequently against the basic data or by setting up automated systems to detect abnormal changes.
Agreements on the methodology of the computation should include such things as item selection, weighting of components and timing of col- lection, as well as the mathematical construction of the index.

The index should also be made available to the central oﬃce with supplementary brieﬁng and explana- tions for price movements.

Any potential problems, such as the need to resample where previously quoted items are no longer available, should be discussed with the national statistical institute in advance.
A con- tinuous quality control may take the form of a recon- ciliation analysis against other related data (including announced price changes) and identifying outliers when compared to previous values of the index.

Data and prices published by another organization or government body may provide a useful comparator.
Where prices are taken over the telephone, it is highly recommended that all price quotations are subsequently conﬁrmed in writing to ensure that any queries may be resolved and that an audit trail is maintained for future months in case of cannot be resolved.

6.75 In all instances it is important to check on a regular basis that the item or service being provided has not changed in any respect, because if this is the case a quality adjustment may need to be applied.
In the case of supermarkets and other large suppliers of data, con- ﬁrmation should be sought from head oﬃce that code numbers have not changed to ensure that the items being priced do not change unexpectedly between one price collection period and the next.

subsequent discrepancies that 6.76 As stated previously, the frequency of collection depends on both the range of prices being monitored and when the prices are known or expected to change.
For instance, bus and rail fares may change once a year on a prespeciﬁed date.

In other instances, prices may change throughout the year as diﬀerent providers review their pricing structures, but the expectation may be that prices will show little volatility.

For example, it may be neces- sary only to contact health insurance companies on a quarterly basis or local authorities for prices of school dinners only at the start of each term.
Decisions on these issues will need to be based on knowledge of local cir- cumstances, with satisfactory procedures in place to detect any change in procedures.

6.77 The number of price quotes required at each collection will depend on individual circumstances and will need to take account of the weights and homo- geneity of the index as well as the underlying volatility of prices (see Chapter 5).
It is also best to avoid situations where a handful of price quotes from, say, an individual retail chain represents a large weight in the index.

The number of prices collected centrally should where pos- sible reﬂect the importance of that item in the shopping basket and the range and volatility of the prices.
6.78 All the data collection principles above are to be followed for all central and head oﬃce price collec- tion, regardless of whether these forms of collection have been introduced for reasons of practicality, cost- eﬀectiveness, or special methodological concerns.
6.79 Further examples of items that may be collected centrally include: some aspects of transport, such as tolls for bridges; situations where there may be a variety of diﬀerent outlets but where there is uniform pricing PRICE COLLECTION for all consumers; and instances where the data requirements for quality adjustment are better met by exploiting a single data source.
By way of elaboration, if none of the towns or cities selected for local price collections have tolls on roads, bridges or tunnels, then these could be unintentionally excluded from the index; but by selecting a sample of these across the country – with prices collected centrally – the index remains representative of these types of expenditure.
Similarly, if the prices of goods and services are the same all over the country, regardless from whom they are bought (for example, newspapers and magazines), then these prices are most cost-eﬀectively collected cen- trally.

The more complex methodological calculations of prices, including quality adjustments, may also be best collected centrally.
Examples of these include some housing costs, and computers and cars (where informa- tion on technical speciﬁcations at the level of detail required for quality adjustment may not be available from shopkeepers).

Price reductions 6.80 One of the principles relating to consumer price indices, which is applied with few exceptions (such as owner-occupier housing costs), is that only transaction prices, that is prices actually paid by individuals or households, should be included in the index.
This may diﬀer from the advertised price if, for example, a dis- count is oﬀered.

In practice, however, discriminatory discounts, which are available only to a restricted group of households (as opposed to non-discriminatory dis- counts that are available to all), are generally excluded on principle.

For example, money-oﬀ coupons and loyalty rewards for previous expenditure are normally ignored and the non-discounted price is recorded.
Also, it may be diﬃcult to obtain the price paid if this is subject to individual bargaining.

It may therefore not come as a surprise that, while the general rule above may appear simple, there are a number of instances requiring special treatment either because of conceptual issues or because of practical diﬃculties.

The following guidelines reﬂect practices followed by a number of countries.
They do not represent a set of rules because the appropriate practice to be followed will be deter- mined by individual circumstances, which might vary between diﬀerent countries.

6.81 Discounted prices should only be taken if gen- erally available to anyone with no conditions attached; otherwise the non-discounted or unsubsidized price is recorded.

In particular, the general practice is to ignore money-oﬀ coupons and loyalty rewards.
A judgement needs to be made, however, relating to the interpretation of ‘‘generally available’’.

For instance, reduced prices for payment by direct debit may be taken into account depending on the extent to which consumers as a whole have access to and use such a service.

A judgement is required in the latter case on the threshold to be set for access, above which action is taken for inclusion in the index.
Alternatively, diﬀerent payment methods may all be priced individually (for example, separate data col- lection for electricity payments by cash, direct debit and 91 CONSUMER PRICE INDEX MANUAL: THEORY AND PRACTICE pre-payment) and weighted together to form a single price index for that item.

6.82 Price discrimination.


Discounts available only to a restricted group of households should be dis- regarded because they are discriminatory, unless they are signiﬁcant and are available either to the vast majority of the population or to identiﬁable subgroups who qualify for such discounts on the basis of demo- graphic or other characteristics not requiring action by the individuals concerned at the time of purchase.
In the latter case, they should be treated as stratiﬁcation or coverage issues in item sampling.

Some judgement is required.

Examples of allowable price discrimination may include lower prices oﬀered to pensioners (for example, discounted travel or haircuts) and discounts for people who receive state beneﬁts.

Another example of a case where prices are not universally available to all, and where judgement is required, is where a nominal or token membership fee is required by the retail outlet.
In these cases, the take-up of such membership – which is widely available to all – needs to be considered in terms of thresholds and general spending patterns of the consumers and the conditions placed on membership which may make the latter restrictive (for example, minimum levels of purchase).

Ease of access to the outlets in question may be a relevant factor as well, say, if in practice the customers need to have the use of a private car.
6.83 Sale or special oﬀer prices should be recorded if these are either temporary reductions on goods that are likely to be available again at normal prices, or are stock-clearing sales (such as January sales or summer sales).

Before designating a price as a ‘‘sale’’ price, however, special care should be taken to ascertain that there is a genuine sale with price reductions on normal stock.
On occasion, stock is continually sold below the recommended retail price or advertised as a special oﬀer even though these prices are available all year.

In such cases, prices should not be considered as sale prices, but can still be collected.

Special purchases of end-of-range, damaged, shop-soiled or defective goods should not normally be priced, as they are likely not to be the same quality as, or comparable with, goods previously priced and are unlikely to be available in future.
If the special oﬀer is limited to the ﬁrst customers, the item should not be priced, as the oﬀer is not available to everyone.

Introductory special oﬀers may be included if they are available to all.

In reality, however, given the need to price the same ‘‘basket’’ each month, such oﬀers will not be chosen as representative items unless they are intro- duced at the time of an update of the ‘‘basket’’ or when a replacement item needs to be chosen.
Discounts on goods close to expiry dates should be disregarded or treated as speciﬁcation or quality changes.

6.84 Bonus oﬀers, extras and free gifts.

Prices for items temporarily bearing extra quantities (for example, 30 per cent extra free) should not be adjusted to take account of the increased quantity if it is thought that the extra quantities involved may not be wanted by most consumers, will not have inﬂuenced the decision to purchase or will not be consumed.
Similarly, free items with other purchases (such as buy 2 get 1 free or free gift 92 with every product purchased) should be disregarded.

Money-oﬀ coupons for future purchases should be dis- regarded, as these may not be used or wanted.

Free gifts such as plastic toys in cereal boxes should be ignored because they are not included in the list for price observations; it is the price to be paid to get the cereal in the box that is relevant.
Collectors should be aware that temporary ‘‘special oﬀer’’ weight changes (X per cent extra free) could become a permanent weight change (for example, cans of alcoholic drinks changing size from 440 ml to 500 ml) and should feed the information back to head oﬃce as they become aware of it.

In this way, head oﬃces can issue new or amended guidance to price collectors about item speciﬁcations.

the price In general 6.85 Stamps.


Sometimes purchasers are given special stamps, which can be accumulated and subsequently exchanged for goods and services.

If a discount is available as an alternative to such stamps, then the dis- counted price should be recorded.

Otherwise, the stamps should be disregarded.

6.86 Trade-ins.

reduction obtained by trading in an old item (for example, a car) compared with the nominal full price should be ignored.
This treatment follows convention, as the transaction essentially relates to a second-hand good and only the service charge levied by the outlet in buying and selling the good comes under the scope of the index.

In reality, however, the situation is not so clear-cut.

For instance, a garage may well give a discount which is greater than the retail value of the traded-in car and, therefore, in eﬀect gives a genuine discount on the new car.
In many cases, discounts from trade-ins are very diﬃcult to evaluate.

The trade-in value may be negotiable in each case, and the full nominal price – which is used as the benchmark against which the discount is measured – may not be known.

It may therefore be best to report the list price or asking price.

6.87 Sales taxes.
6.87 Sales taxes.

When an indirect tax is not included in the price of individual items in a shop, but is instead added on when the customer pays for the item, great care must be taken to record the price including tax.
To make sure of this, with items for which the price is normally quoted pre-tax, and in areas where a gen- eral sales tax is added to the bill, the price collection forms should require the collector to indicate whether or not the price recorded does include the tax – as a price check – so that it can be added where necessary.

6.88 Tips for services.


If a compulsory service charge is included, for example on a restaurant bill, only the compulsory amount should be included in the price, but not any additional discretionary tips.

For services which are free in principle, but which in practice can rarely be obtained without what amounts to a tip, or where tip- ping at a standard rate is the common practice, such tips should be added to the speciﬁed price.
6.89 Regular rebates or refunds should only be taken into account when attributable to the purchase of an individual identiﬁable product and granted within a time period from the actual purchase such that they are expected to have a signiﬁcant inﬂuence on the quantities buyers wish to buy.
For example, money-back deposits on bottles should be deducted from the price if they are a suﬃcient incentive for returning the bottle, while money- back oﬀers on lawn mowers after a ﬁve-year period should be disregarded.

In all cases, a consistent decision for each item must be applied over time.

Decisions about the treatment of rebates are not easy to recommend, as many decisions are made on an individual basis.
They may reﬂect income rather than expenditure changes and may require diﬀerent treatment for, say, national accounts uses.

6.90 Irregular rebates or refunds should only be taken into account when they apply to the purchase of an individual product and are granted within a time period such that they are expected to have a signiﬁcant inﬂuence on the quantities purchasers are willing to buy.
Loyalty rebates or coupons associated with previous expenditure at the outlet, to be used for similar or other purchases, should generally be disregarded, as they are discrimi- natory.

If they are signiﬁcant factors, they should be treated as stratiﬁcation or coverage aspects of sampling (see Chapter 5).

One-oﬀ rebates (for example, associated with privatization) should be disregarded as they do not relate to the speciﬁc time period of the consumption and are unlikely to aﬀect levels of consumption.
They can be viewed more as a source of additional income.

6.91 Credit card and other payment arrangements involving interest, service charges or extra charges incur- red as a consequence of failing to pay within a speciﬁed period of time from the purchase should be disregarded.

For example, zero interest as well as positive interest loans granted to ﬁnance a purchase should be dis- regarded when determining the price.
Reductions for cash payments may be included but care should be taken to ensure consistent treatment from one period to the next.

Price bargaining 6.92 Bargaining relates to a situation where prices are individually negotiated between sellers and pur- chasers, and are not predetermined.
The process of negotiation is a characteristic of, for example, market- places in many African countries where almost every- thing to be purchased must be negotiated to arrive at an agreed price, including a wide range of life’s daily necessities that can account for a large part of household consumption.

The system of bargaining is characterized by its great ﬂexibility in the setting of prices.
Final trans- action prices and quantities will vary from one trans- action to another and cannot be determined until the purchase has been made.

Similarly, there will be vari- ations between transactions in the quality of the goods being purchased.

Clearly, these special conditions re- quire special methods to determine purchasers’ prices for inclusion in the consumer price index.
6.93 It can be argued from the viewpoint of the system of national accounts that bargaining is a form of price discrimination.

A purchaser is not free to choose the purchase price because the seller can charge diﬀerent categories of purchasers diﬀerent prices for identical goods and services sold under exactly the same cir- cumstances.
It follows that ‘‘identical’’ products sold at diﬀerent prices should be recognized as having the same PRICE COLLECTION quality, and their prices must be averaged to obtain a single price relative to calculating price indices.

In re- ality, the variation in transaction price can rarely be associated with identiﬁable price-related categories of customers.

Rather, purchasers may inadvertently buy at a higher price than may be found elsewhere or could have been ﬁnally negotiated.
Notwithstanding this, collectors of prices should guard against the presump- tion that price diﬀerences do not relate to quality (or quantity) diﬀerences.

6.94 Where prices are determined by bargaining, standard price survey methods – which consist of col- lecting prices directly from sellers – can generate erratic price indices that do not reﬂect actual price movements in a market.
For example, prices collected by enumera- tors depend on their ability, willingness and power to bargain, in the same way as actual prices paid by genuine purchasers.

In addition, prices can vary during the course of a day as well as from one day to the next, adding an extra dimension to the concept of repre- sentativeness.

A number of survey methods and price collection techniques have been developed to overcome the diﬃculties inherent in measuring prices that have been bargained.
6.95 Survey by purchase of products.

The principle is that price collection should be carried out in conditions that simulate as closely as possible situations in which real transactions actually take place.

Price collectors behave like regular purchasers by actually purchasing items to be priced and spreading their purchases over the day to ensure representativeness.

In each case, the ﬁeld manager will need to carry out regular checks on quantities and prices obtained by collectors.
The fol- lowing approaches may be taken: Price collectors buy items to determine the relevant price through bargaining.

They should be trained to behave as normal purchasers and strive to get the lowest possible price from selected outlets and sellers.

Given the high turnover of sellers, the sample of sellers should be partially renewed on a regular basis to ensure that it remains representative and chained in as appropriate.
Price collectors buy items and, in addition, are given an incentive to get the best price.

For example, a price ceiling may be set and the collector may receive a proportion of the diﬀerence between the ceiling and the bargained price.

This incentive system guards against potential diﬃculties caused by the collector not getting the lowest price because, unlike an ordi- nary customer, he or she is not concerned with max- imizing value for money and is not constrained by income.
6.96 Survey of purchasers.

The prices purchasers have paid are collected throughout the day immediately after the purchaser leaves the outlet or market stall, together with a record of the quantity and quality of the product purchased.

The extent of the haggling should be determined (for example, opening and closing prices) together with an indication of the relevant parameters determining the price.
A form of incentive payment for survey participation may be needed where there is 93 CONSUMER PRICE INDEX MANUAL: THEORY AND PRACTICE reluctance among purchasers to submit to such time- consuming questions.

6.97 For the survey by purchase of products and the survey of purchasers, all items in the basket of items used to calculate the consumer price index that are subject to bargaining should be covered.
The number of prices collected needs to be suﬃcient both to cover all relevant items and to provide a reliable guide to average price.

This may be diﬃcult to determine beforehand, although previous price collections should provide some guidance.

It is suggested that price collectors engaging in a survey of purchasers are given a form on which to record the number of quotations per stall or shop, as indicated by the various respondents.
This can be used to check the number of quotes obtained against the target number set by head oﬃce.

An example of such a form is given in Table 6.1.

6.98 Survey of trends in wholesale prices.

A limited parallel collection of wholesale prices can be a useful supplement for problematic items where the information obtained from the above survey techniques is only par- tially successful, for example where there is a deﬁcit in the number of observations obtained.
Ideally, prices should be obtained from the particular wholesalers where the relevant retailers get their goods.

All factors should be observed which might result in increases in the corre- sponding retail prices, such as changes in taxes on retail activities, licence fees and the rental for the market stall.

Assuming that those factors remain constant over time, the evolution of wholesale prices may be used as a proxy for the retail price index of relevant items.
The price of an item for the current period would be estimated by mul- tiplying the price of the previous period by the corre- sponding evolution in wholesale price.

6.99 Determination of the prices paid by a purchaser can be problematic where the ﬁnal price is for a bundle of items, for example where a stall holder gives the purchaser extra quantities as a bonus for buying a num- ber of goods.
If the bonus comprises several categories of items, including the item on which a transaction price was being directly negotiated, then the purchase has to be split into as many sub-transactions as item categories.

In these cases, a commonsense approach is needed.

There is a ﬁne dividing line between this type of circumstance and the ‘‘two-for-one’’ oﬀers sometimes found, for exam- ple, in Western-style supermarkets.
The latter form of discount is often excluded from price calculations on the basis that the purchaser does not want or use the addi- Table 6.1 Example of a survey form showing the number of price quotations by shop or stall Actual number of quotations Shop/Stall 1 Shop/Stall 2 .

.

.. Shop/Stall n 0 4 5 7 3 5 8 2 5 4 8 6 Targeted number of quotations (set by head ofﬁce) 5 4 8 5 Items Item 1 Item 2 Item 3 .

.

.

Item k 94 tional amount supplied.


Additional perishable goods, for example, will become out of date and be thrown away.

This argument is less relevant in market purchases in a developing country, where many consumers will be living on a subsistence income and therefore will consume all purchases.

In such cases, purchasers will have actively bartered an overall price for the total basket of pur- chases, including any ‘‘free’’ goods thrown in.
6.100 The method for determining the price paid by the purchaser is illustrated in the following example: a purchaser wants to buy 5 kg of carrots and is oﬀered a bonus consisting of 500 grams of carrots, 100 grams of lettuce and 200 grams of baby marrow.

6.101 Three transactions can be identiﬁed, involv- ing: 5.5 kg of carrots; 100 grams of lettuce; and 200 grams of marrow.

The bonus has to be valued at prices at which the seller would have sold and the purchaser would have bought the items.
The assumption made is that prices, in local currency units (LCU), would have been determined through bargaining on the same con- ditions as the price of the item needed (carrots).
If the opening value of 5 kg of carrots is LCU 15,000 and the closing value LCU 12,000, whereas the opening values of other foodstuﬀs included in the bonus are LCU 990 for a bunch of 264 grams of lettuce and LCU 4,620 for a heap of baby marrow of 4.4 kg, the actual closing price of carrots will be determined as shown in Table 6.2.

The actual purchaser’s price of carrots is found to be LCU 2.0967 per gram or LCU 2,096.7 per kilo.
6.102 If the price collector does not know the closing price at which lettuce and baby marrow would have been sold by the seller of the carrots, then it can be estimated.

This is done by collecting opening values and standard quantities from a sample of sellers in the same market or at diﬀerent outlets in the same area.

The average opening price of an item is equal to the sum of opening values of the item divided by the sum of relevant standard quan- tities.
For each bonus item (lettuce and baby marrow), the resulting average opening price will be divided by the bargaining ratio calculated on the item needed (carrots) to estimate a closing price for that bonus item.

The value of each bonus item is obtained by multiplying the closing price by the quantity oﬀered.

If the packet of bonus items contains an item of the same quality as the requested item, that bonus item will be valued on the basis of the closing value of the requested item.
Forced replacements, product substitution and quality adjustment 6.103 A diﬃculty which confronts both local and central price collections occurs when an item that was being priced is no longer available and a substitute needs to be found.

This is brieﬂy discussed here because it relates to real decisions facing price collectors in the ﬁeld, but the issues are covered in more depth in Chapters 7 and 8.
In cases where a replacement has to be found, the price collector should normally take the nearest equivalent product available in the outlet, taking into account those characteristics which will be most inﬂuential in determining price and purchasing habits (for example, one out-of-date or obsolete item should Table 6.2 Example illustrating the method for determining the actual price paid by the purchaser when bargaining takes place Requested item Bonus items PRICE COLLECTION Carrots 15 000 5 000 3 3 500 1 500 1 200 2.4 1.25 Lettuce Baby marrow 990 264 3.75 3.75 100 375 300 3 1.25 4 620 4 400 1.05 1.05 200 210 168 0.8 1.25 Opening value of standard/requested quantity (local currency units) Standard/requested quantity (grams) Opening unit price of standard/requested quantity (local currency units per gram) Opening unit price of bonus quantity (local currency units per gram) Bonus quantity (grams) Opening value of bonus quantity (local currency units) Carrots 15 000 5 000 3 Closing value of items received (local currency units) 12 000 New price (local currency units per gram) Bargaining ratio Payment (local currency units) Estimated closing value of bonus (local currency units) Actual value of requested item (all carrots) (local currency units) Quantity received of requested item (grams) Actual purchaser’s unit price of requested item (local currency units per gram) Improved bargaining ratio 1(12000300168) 7 5500 = 2.0967.
23 7 2.0967 = 1.52.

2.4 1.25 12 000 1 668 10 332 5 500 2.09671 1.522 not be replaced with a close item which may also shortly suﬀer the same fate).

Nevertheless, where it is considered desirable to take the opportunity provided by product substitution to update the sample, a ‘‘most representa- tive’’ replacement may be chosen.

In the latter case, care must be taken to ensure that suﬃcient controls are in place to achieve the desirable end.
6.104 When a replacement is made, it is important for the price collector to provide a detailed speciﬁcation of the new item so that head oﬃce can identify any associated quality change.

This is to ensure that the consumer price index continues to reﬂect the cost of buying a ﬁxed, constant quality basket of goods.

Head oﬃce should then use the information collected to decide on any relevant quality adjustment to be applied.
6.105 When such a situation occurs, a nominal price in the base month (which for some indices will be the previous month) is needed for the new or replacement item.

The latter may be obtained from the shopkeeper or one of three methods can be applied to take account of quality diﬀerences, which can then be used to estimate a new base price.

These are direct comparison (that is, when there is no change in quality), direct (explicit) quality adjustment, or indirect (implicit) quality adjust- ment.
When a new rather than a comparable replace- ment item is priced, it may be necessary for the new item to be kept out of the index for a short period until there is suﬃcient evidence of its longer-term availability and price stability.

6.106 In some countries, a table of quality coeﬃ- cients is used to adjust prices.
In one North African country, for example, the item ‘‘green tea’’ should be represented by Minara tea; however, if this is unavail- able an alternative tea may be collected and that price scaled by the relevant coeﬃcient (for example, Oudaya tea 1.20).

More detailed guidance on direct and in- direct quality adjustment is given in Chapter 7.
6.107 If an outlet closes or refuses to allow further price collection, then another similar outlet should be selected from the same location and the indirect quality adjustment approach used to calculate new base prices.

See Chapter 5 on sampling for replacing outlets within locations.
Related issues Electronic reporting 6.108 Electronic reporting for centrally collected prices and use of hand-held computers for local price collection can introduce greater eﬃciency into price collection and processing, as well as providing more scope for eﬀective auditing, but both are dependent on the introduction of eﬀective quality control procedures.

Electronic reporting through the use of electronic point of sale (EPOS) or scanner data is also likely to increase over time.
6.109 Electronic reporting for centrally collected prices.

Centrally collected data can be collected electronic- ally in a number of ways.

Once initial contact has been made with data suppliers, a mutually convenient 95 CONSUMER PRICE INDEX MANUAL: THEORY AND PRACTICE electronic data collecting procedure can be initiated.
Options include: – emailing data collection spreadsheets between the national statistics institute and the retailer; – emailing of price lists at agreed times by retailers; – touch-tone dialling facilities for data to be supplied in an agreed format; – use of the Internet (supplemented if necessary by tele- phone calls to clarify deﬁnitions and availability).

6.110 Hand-held computers.


The greatest gains from the use of hand-held computers for local price collection are likely to be drawn from eﬃciency in data transmis- sion, better quality data as a result of the additional editing facilities available in the ﬁeld, and the elimina- tion of transcription errors.

In addition, hand-held computers can generally speed up timetables.
6.111 The validation checks made during local price collection using hand-held computers will generally diﬀer very little from those that should be carried out at central oﬃce when paper forms are received under the more traditional methods of collecting price data.

The advantage of hand-held computers is that they provide the opportunity to validate prices in the ﬁeld and in consequence correct errors at the time of price collec- tion, rather than attempting to do so afterwards.
In practice, it may be expensive and very diﬃcult to check prices after collection.

For example, prices may have changed in the intervening period and the price collector may have to rely on the shopkeeper’s memory.

6.112 The choice of hand-held computer will depend on a number of factors, including price, reliability, maintenance and ease of use.

The computing functions of data transfer, including back-up and downloading of data, as well as compatibility with oﬃce systems, are also important.
Other considerations of particular con- cern to the price collector include ergonomic aspects, size and weight, editing facilities, and expected battery life.

The risk of theft and other security matters will also play a part.

6.113 The introduction of hand-held computers can involve a signiﬁcant initial outlay associated with pur- chasing the computers, developing the software and training price collectors.

In addition there will be on- going maintenance costs.
These costs can sometimes be reduced or spread either by using the machines for other data collection in the national statistical institute, for example a household budget survey, or by contracting out to another organization that may already deploy these machines for other statistical surveys.

These costs can be oﬀset, at least in part, by more eﬃcient working by price collectors and savings generated by less tran- scribing and inputting of data by hand, and a reduction in data editing by head oﬃce staﬀ.
6.114 Careful planning is required in moving from a paper-based collection system to one using computers in order to avoid the risks inherent in such a change.

National statistical institutes planning a move to price collection using hand-held computers should embark on extensive pilot testing and should also consider some limited double-running in parallel with the old paper collection system to ensure the robustness of the new 96 method and that it is producing the same numerical results.
6.115 The additional facilities oﬀered by hand-held computers, including local editing of prices, plus the elimination of the need for data transcription, may necessitate a general reorganization of the process for producing the consumer price index, and a redeﬁning of roles and interaction between diﬀerent members of the production team and between head oﬃce and price collectors.
6.116 It is important that clear rules and procedures are set out controlling the changes that can be made in the ﬁeld by the price collector and the changes that should be made centrally.

For example, replacement outlets could be pre-programmed in the event that out- lets close down or refuse entry.

Flexibility should allow price collectors to select and key in the new attributes for replacement items subject to procedures controlled cen- trally.

6.117 EPOS or scanner data.


Electronic point of sale (EPOS) data usually refers to data obtained directly from a retailer’s electronic point of sale, while scanner data usually refers to a commercial database that collates individual EPOS data.

National statistical institutes are increasingly looking towards EPOS or scanner data as a convenient method of obtaining up-to-date and accurate information, not only on the quantity and prices of goods sold but also on their speciﬁcation.
The latter can be used to control the representativity of the sample and also to measure changes in quality.

The advantage of this is that data are collated electronically without the necessity of sending price collectors into the ﬁeld.
6.118 When considering the use of scanner data, account needs to be taken of such matters as the repre- sentativity of outlet and product coverage, and also whether the average prices given in scanner data accu- rately reﬂect actual transaction prices in the outlets themselves.

In addition, it cannot be assumed that the geographical and population coverage or the treatment of goods and transactions matches the scope of the index.
Scanner data are also likely to be of little use in collecting prices of services, which in many countries comprise an increasing share of transactions and thus of weights in consumer price indices.

On a practical front, the unique identiﬁcation of products can sometimes be problematical, as one item might be covered by more than one code number, and code numbers may not be uniquely assigned to one product and may be recycled as items disappear.
Purchasing power parities 6.119 Purchasing power parities are used to deﬂate major economic aggregates, such as gross domestic product, to enable intercountry comparisons of real income levels to be made in terms of real volume, that is, adjusted to account for local prices and diﬀerent con- sumption patterns.

Purchasing power parities consist of intercountry comparisons of prices for a basket of goods and services that is both representative of and compara- ble between the countries involved.
The underlying price data therefore diﬀer from those used in consumer price PRICE COLLECTION indices in so far as the latter basket is designed to be representative solely of private household consumption in the economic territory of an individual country.

6.120 It would be attractive in principle to construct consumer price indices and purchasing power parities from the same basic set of price data.

In practice, the scope for this may be limited because of the diﬀering objectives of the two exercises.
In particular, the addi- tional need for prices collected in the context of purchas- ing power parities to be comparable between countries will generally result in a more tightly deﬁned basket compared with that likely to be available and used for a consumer price index.

6.121 An investigation of the potential overlap between the two baskets may, however, identify potential areas where one price collection may suit both purposes.
This may be the case particularly for unbranded goods and locally produced fresh fruit and vegetables, for exam- ple, where a locally produced dessert apple of a standard quality may be compared across countries without recourse to a reference to the variety concerned.

In con- trast, branded items – whether food or non-food – may be more problematical because of diﬀerences in avail- ability and speciﬁcation between countries.
6.122 In some cases, scanner data might provide a useful common source of price data for at least some elements of the purchasing power parities calcula- tion, notwithstanding the drawbacks mentioned above.

Annex 4 goes into more detail on issues relating to pur- chasing power parities and the International Compar- ison Program (ICP).
Data quality and quality assurance 6.123 Checks should be carried out to ensure the accuracy of the data on prices and that the index itself has been compiled according to the proper methodology.

Checks to ensure that data are complete and correct should be carried out as early as possible in the collection and compilation processes.
A return to the shop to re- input prices becomes increasingly less feasible as time goes on, and there is a greater risk that the prices in the shops will have changed since the initial collection.

It is not possible to prescribe the type and range of checks that should be carried out.

The checks will depend on individual circumstances, including sample design and the medium used for the collection of prices.
For exam- ple, the use of hand-held computers by price collectors facilitates much more detailed checking at the time of the initial collection of prices in the shop than the equivalent paper system.

Further guidance on quality assurance is given in Chapter 12.

Documentation 6.124 The importance of good documentation can- not be overemphasized.

Documents are needed to explain what is to be done, when it should be done, how it should be done and why it should be done.
Preparing such documents provides a useful opportunity to ensure the quality of current procedures used to collect prices and compile the index.

It also provides an opportunity to review and improve these procedures.

Once in place, documentation serves two purposes in the context of producing the index.

First, it enables somebody to take over the work if the person responsible falls ill or leaves.
Second, it provides a quality check to ensure that the procedures that should be carried out are indeed being carried out in practice.

More generally, documentation can provide a useful reference for users of consumer price indices.

Documentation is discussed in more detail in Chapter 12.

97 .

n o i t a i r a v l a n o s a e S .

s n e e u Q n a i t p y g E h t n o m t s a L .

m e t i l e b a r a p m o C .

e c i r p f l a H .


r e f f O l i a c e p S ) e t a i r p o r p p a s a s e g a s s e m d e d o c - e r p .

 / t e l t u o r o f y r a s s e c e n s a l e t e p m o c ( .

3 £ g 0 0 4 p 9 5 p 9 8 .

t a f w o l , t u c m u m e r P i .


 , , w e n – s e o t a t o P e k a m r o d n a r b : n o i t p i r c s e D t e l t u o l i a t e R m e t I .

d e r u t c p i a z z p i h t i w i .

g n g a k c a p e u b l d n a d e R x o b d e R .

t s a e F t a e M .
t s a e F t a e M .

 – 0 0 3 i e z s m u d e m i , a z z p i n e z o r F .

l e b a l e u b l h t i w e l t t o b c i t s a P l .

 , d e z i r u e t s a p , k l i M .

f f o t n e c r e p 5 2 m o r f y r e v o c e R i , e c a P l ‘ ‘ s a d e v r e s l y s u o v e r P i .

’ ’ d a a s l d n a i s p h c .


’ ’ s d a e h i g n k a T l ‘ ‘ l , y s u o v e r P i R C N 0 0 .

0 3 £ 0 5 .

7 £ 0 0 .

2 1 £ .

l l i t r a e N i .

a n h C n i e d a M .

s e o s l r e h t a e l d n a s r e p p u r e h t a e L .

u n e m n a M i .

d a a s l d n a i s p h c , d o C .

’ ’ i e k e e S l ‘ ‘ e m a n e o h S .

s e o h s t r u o c k c a B l .


t e k c a p n o h s a ﬂ e u B l s p e t S p U l l i F , e s r u o c i n a m , l a e m t n a r u a t s e R y f i c e p s , l a e m g n n e v e i i n o h s a f , s e o h s ’ s n e m o W ) s r u h T – n o M ( y a d k e e W l .

 , g n n e v e i , n o s s m d a i e r t a e h T .

e c n a m r o f r e p i g n n e v e t l u d a , s l l a t s t n o r f .
w e n = N ; y r e v o c e r = R ; y r e u q = Q i ; t h g e w = W ; r e ﬀ o l i a c e p s r o l e a s = S l ; e b a r a p m o c = C 1 l .

d n a g n E n i e d a M .

r e t s e y o p l t n e c r e p 5 2 .

e i t e e r f s e d u c n I l Q 9 9 .

4 3 £ , n o t t o c t n e c r e p 5 7 .

e t i h W .
e t i h W .

d n a r b ’ ’ s t r i i D e k c D i ‘ ‘ n e M r o F l a m r o F d e v e e s l g n o l , t r i h s l a m r o f ’ .

e l b a c i l p p a n i n e h w k n a l b ’ ’ e k a m r o d n a r b ‘ ‘ g n i v a e l , : s e t o N .
t e l t u o h c a e : e m a n t e l t u O : e m a n s ’ r o t c e l l o C : .

6 x i d n e p p A 98 ADJUSTING FOR QUALITY CHANGE 7 Introduction 7.1 The measurement of changes in the level of con- sumer prices is complicated by the appearance and dis- appearance of new and old goods and services, as well as changes in the quality of existing ones.
If there were no such complications, then a representative sample could be taken of the items households consume in period 0, their prices recorded and compared with the prices of the same matched items in subsequent periods, say t. In this way the prices of like would be compared with like.

However, such complications do exist.

For example, an item may no longer be produced in period t+1, so its price comparison cannot be undertaken between periods 0 and t+1.
7.2 A number of methods are available to remedy this.

A replacement item may exist in period t+1.

If it is of the same quality, its price can be compared with the ‘‘old’’ item’s price in period t. But the replacement item may well be of a diﬀerent quality.

One option is to ignore the quality diﬀerence and continue comparing the price of the ‘‘new’’ replacement item in t+1 with that of the old one in period t to continue the series.
An adjustment for the diﬀerence in quality is still being made; it is just that it is a very poor adjustment, because the change in quality has no eﬀect on the price.

A second option is to exclude from the index those items for which quality changes, and to compile the index link between t and t+1 only for matched items having characteristics that are the same.
This exclusion amounts to an implicit quality adjust- ment, one that assumes the overall price change of existing matched items will be the same as the quality- adjusted price change between the missing old and replacement new items.

In reality, however, price chan- ges generally vary over the stages of a product’s life cycle.

Price changes at the time of, say, a model’s upgrade – when an item is missing and replaced – may be quite from those at other stages.
The implicit diﬀerent assumption may therefore be inappropriate.

Third, the price change of a new replacement item may be spliced onto the index if the prices of the disappearing and replacement items are available in a common overlap period, say period t. The old item’s price change between periods 0 and t is multiplied by the replacement item’s price change between periods t and t+1.
Yet again, there is an implicit quality adjustment, one that requires the price diﬀerence between the old item and its replacement in period t to reﬂect the eﬀect of the quality diﬀerence on price.

Such diﬀerences may also be in part the result of strategic price-setting behaviour related to the period in the item’s life cycle.
7.3 There are other methods of adjusting the prices of non-comparable replacements for quality diﬀerences, including ones that use explicit estimates of the eﬀect of the quality change on price.

There are a number of methods of deriving such explicit estimates, and the suit- ability of explicit quality adjustments depends as much on the method used as on the availability of appropriate data to implement the method.
In each case, whatever procedure a statistical oﬃce follows, a quality adjust- ment to prices is made in every period when an item is not available.

The purpose of this chapter is to help ensure that these quality adjustments are the appropriate ones.

7.4 There are three main reasons for considering how to adjust for quality change.

First, the scale and pace of innovations are substantial.
Second, methodological there is a lack of consistency in the methods chosen by statistical oﬃces for dealing with quality changes; thus comparisons of consumer price indices between product areas, across countries, and over time may be misleading.

Finally, a number of empirical studies on the eﬀects of using diﬀerent methods found that choice of method does indeed matter (Dulberger, 1989; Armknecht and Weyback, 1989; Moulton and Moses, 1997; Lowe, 1996).
7.5 Against these concerns, it must be recognized that statistical agencies do guard against quality changes by using the matched models method.

Price collectors record the features of selected items and collect prices for the very same models in subsequent periods in order to compare like with like.

If a product group exists in which there are no items whose quality changes and no new or disappearing goods and services, then the matched models method based on representative items works.
More generally, three potential sources of error arise from the matched models approach: missing items, sample space change, and new products.

Why the matched models method may fail 7.6 The long-run price change for an item is mea- sured by comparing the price of the item in the current period with that in the price reference period, the period in which it, along with most other items, entered the sample.
Missing items 7.7 The ﬁrst source of error, and the focus of this chapter, is when an item is no longer available in the outlet.

It may be discontinued or it may not be available to the same speciﬁcation – its quality has changed – and it is eﬀectively missing in the current period.

The item’s 99 CONSUMER PRICE INDEX MANUAL: THEORY AND PRACTICE price may be missing for other reasons.
It may be a seasonal item or one whose price does not need to be recorded so frequently, or it may be that the item is a custom-made product or service, supplied each time to the customer’s speciﬁcation.

7.8 It is necessary to distinguish between items that are permanently and temporarily missing.

Items that are temporarily missing are items not available and not priced in the month in question, but that are priced in subsequent months.
The items may be missing because, for example, demand is seasonal, as is the case with some fruits and vegetables, or there are shortages.

Some commodities are priced on a less frequent basis, maybe quarterly or biannually, because their price changes are irregular.

They are therefore missing when they are ‘‘oﬀ cycle’’.

7.9 The concern with seasonal items is to impute their missing prices until the item reappears.
The impu- tation methods used are similar in some cases to those used for quality adjustment.

The temporary nature of the imputation, however, requires that they be separately identiﬁed by the respondent as ‘‘temporarily missing’’ or ‘‘seasonal’’.

Principles and methods for such imputations are outlined by Armknecht and Maitland-Smith (1999) and Feenstra and Diewert (2001), and in Chapter 22.
The concern in this chapter is with permanently missing items and with making imputations on a continuing basis or using a replacement item.

7.10 A number of approaches are available for dealing with missing items: The item may be dropped on the assumption that the aggregate price change of a group of other items reﬂects change in the missing item – an implicit quality adjustment to price.
A replacement item may be selected and the replace- ment item’s price may be used for the comparison because the replacement is deemed to be comparable in quality to the missing item.

 The replacement may be deemed to be non- comparable with the missing item, but prices on both the missing and replacement items may be available in an overlap period before the former item was missing.
The price diﬀerence in this overlap period may be used as an estimate of the quality diﬀerence to quality- adjust the replacement item’s price.

 The replacement price of a non-comparable replace- ment may be used, with an explicit estimate of the adjustment for the quality diﬀerence to extricate the ‘‘pure’’ price and quality change.

7.11 In many cases, therefore, there is a need to make a quality adjustment to the replacement item’s price.
A quality adjustment in this instance is an adjustment to the price (price change) of the replacement item (com- pared with the missing item) to remove that part of the price change that results from quality diﬀerences.

A quality adjustment can be taken to be a coeﬃcient that multiplies the price of, say, the replacement item to make it commensurate, from the consumer’s point of view, with the price of the original.
7.12 To take a simple example, suppose that the size (or quantity) an item is sold in is a quality feature.

100 if 2 units of Suppose that the size of the missing item and its replace- ment diﬀer.

Assume that a quantity k of the replacement is sold for the same price as a quantity j of the original.

Whether the consumer buys one unit of the original or j/k units of the replacement makes no diﬀerence – they are worth the same.
In order to make the price of one unit of the replacement commensurate with the price of one unit of the original, the replacement must be multi- plied by k/j.

This is the required quality adjustment.

For example, item were equivalent to 3 of the original, the required quality adjustment to be applied to the price of the replacement item is 2/3.
Suppose one unit of the replacement actually sells at the same price as one unit of the original, then the price of the replacement, after adjusting for the change in quality, is only 2/3 that of the price of the original.

If one unit of the replacement sells for twice the price of the original, then the quality-adjusted price is 4/3 that of the original: the price increase is 33 per cent, not 100 per cent.
The consumer price index seeks to record the change between the price of the original and the quality- adjusted price of the replacement.

the replacement 7.13 The approaches listed in paragraph 7.10 will be discussed later in some detail, along with the assump- tions implied by them.

By deﬁnition, the prices of the unavailable items cannot be determined.

The veracity of some of the assumptions about their price changes, had they been available, is therefore diﬃcult to estab- lish.
What is stressed here is that the matching of prices of items allows for the measurement of price changes untainted by quality changes.

When items are replaced with new ones of a diﬀerent quality, then a quality- adjusted price is required.

If the adjustment is inap- propriate, there is an error, and if it is inappropriate in a systematic direction, there is a bias.

Careful quality adjustment practices are required to avoid error and bias.

Such adjustments are the subject of this chapter.


Sampling concerns 7.14 There are four main concerns with regard to sampling.

First, the matching of prices of identical items over time, by its nature, is likely to lead to the monitoring of a sample of items increasingly unrepresentative of the population of transactions.
It may be that the prices of old items being dropped are relatively low and the prices of new ones relatively high, and such diﬀerences in price remain even after quality diﬀerences have been taken into account (Silver and Heravi, 2002).

For strategic reasons, ﬁrms may wish to dump old models, perhaps to make way for the introduction of new models priced relatively high.
Ignoring such ‘‘unmatched’’ models in measuring a consumer price index will bias the index downwards (see paragraphs 7.150 to 7.152 below).

Therefore, in a curious way, the very method of match- ing, used to ensure constant quality, may itself lead to bias by omitting items whose price changes are unusual (see also Koskima¨ ki and Vartia (2001) for an example).

Chapter 8 suggests that the strategy for quality adjust- ment of prices should be linked to one of item selection and chaining.
The strategy is particularly pertinent to sectors with dynamic technological innovations (see also the discussion of hedonic price indices, below).

7.15 Second, because of the additional resources required for quality adjustments to prices, it may be in the interests of the price collectors and desk statisticians, and indeed fall within their guidelines, to avoid making non-comparable replacements and quality adjustments.

Thus items continue to be monitored until they are no longer produced.
This means that old items with limited sales are monitored.

Such items may exhibit unusual price changes as they near the end of their life cycle, because of the marketing strategies of ﬁrms.

Firms typi- cally identify gains to be made from diﬀerent pricing strategies at diﬀerent times in the life cycle of products, particularly at the introduction and end of the cycle (Parker, 1992).
The (implicit or otherwise) weight of end- of-cycle items in the index would thus remain relatively high, being based on their sales share when they were sampled.

Furthermore, new unmatched items with pos- sibly relatively large sales would be ignored.

As a con- sequence, undue weight would be given to the unusual price changes of matched items at the end of their life cycle.
7.16 A third sampling concern relates to the timing of item substitution: when a replacement item is chosen to substitute for an old one.

Instructions to pick a com- parable replacement to avoid messy quality adjustments to prices compound the problem.

Obsolete items are by their nature at the end of their cycles and comparable replacements, to be comparable, must also be near or at the end of their cycles.
Obsolete items with unusual price changes at the end of their cycles are thus re- placed by obsolete items with, again, unusual price changes.

This compounds the problem of unrepresenta- tive samples and continues to bias the index against technically superior items delivering cheaper service ﬂows.
7.17 The ﬁnal sampling problem with the matching procedure is when the price collector continues to report prices of items until replacements are forced, that is, until the items are no longer available, and has instructions to replace those items with typically consumed or popular items.

This improves the coverage and representativity of the sample.

But it also makes reliable quality adjustments of prices between the old obsolete and new popular items more diﬃcult.
The diﬀerences in quality are likely to be beyond those that can be attributed to price diﬀerences in some overlap period, as one item is in the last stages of its life cycle and the other in its ﬁrst.

Furthermore, the technical diﬀerences between the items are likely to be of an order that makes it more diﬃcult to provide reliable, explicit estimates of the eﬀect of quality diﬀerences on prices.
Finally, the (quality-adjusted) price changes of very old and very new items are unlikely to meet assumptions of ‘‘similar price changes to existing items or classes of items’’, as required by the imputation methods.

Many of the methods of dealing with quality adjustment for unavailable items may be better served if the switch to a replacement item is made earlier rather than later.

Sampling concerns can be seen to be inex- tricably linked to quality adjustment methods.
This will be taken up in Chapter 8 on item selection and the need ADJUSTING FOR QUALITY CHANGE for an integrated approach to dealing with both repre- sentativity and quality-adjusted prices.

New products 7.18 A third potential source of error arises when something new is introduced into the marketplace.

It is diﬃcult to distinguish between new items and quality changes in old ones; this diﬃculty will be discussed in Chapter 8.
When a really new item is introduced, there is an immediate gain in welfare or utility as demand switches from the previous technology and other goods.

For example, the introduction of the zip fastener for clothing, instead of buttons, was a completely new good that led to an initial gain in utility or welfare to con- sumers as they switched from the old to the new tech- nology.
This gain from its introduction would not be properly brought into the index by waiting until the index was rebased, or by waiting for at least two suc- cessive periods of prices for zip fasteners and linking the new price comparison to the old index.

Subsequent prices might be constant or even fall.

The initial welfare gain would be calculated from a comparison between the price in the period of introduction and the hypothetical price in the preceding period, during which supply would be zero.
The practical tools for estimating such a hypo- thetical price are not well developed, though this subject is discussed in more detail in Chapter 21.

For a consumer price index built on the concept of a base period and a ﬁxed basket, there is, strictly speaking, no problem.

The new good was not in the old basket and should be excluded.

Although an index properly measuring an old ﬁxed basket would be appropriate in a deﬁnitional sense, it would not be representative of what we buy.
Such an index would thus be inappropriate.

For a cost of living index concerned with measuring the change in expendi- ture necessary to maintain a constant level of utility (see Chapter 17), there is no doubt that it would be con- ceptually appropriate to include the new good.

The nature of quality change 7.19 This section considers what is meant by quality change and then outlines the methods available for dealing with unavailable price quotes.
To understand the ‘‘meaning’’ of quality change requires a conceptual and theoretical platform, so that adjustments to prices for quality diﬀerences are made against a well-considered framework.

7.20 A starting point is to appreciate that over time the quality of what is produced changes.

The example of new cars is used here.

Bode and van Dale´ n (2001) undertook an extensive study of the price measurement of new cars in the Netherlands between 1990 and 1999.
The average nominal price increase over this period was found to be around 20 per cent, but the mix of average quality characteristics changed over this period.
For example, the horsepower (HP) increased on average from 79 to 92 HP; the average eﬃciency of fuel con- sumption improved from 9.3 to 8.4 litres/100 km; the share of cars with fuel injection rose from 51 per cent to 91 per cent; the proportion of cars with power steering 101 CONSUMER PRICE INDEX MANUAL: THEORY AND PRACTICE increased from 27 per cent to 94 per cent; airbags from 6 per cent to 91 per cent, and similarly for central locking, tinted glass and much more.
This churning in the quality mix of what is purchased is one aspect of quality change.

In matching the prices of a sample of models in, for example, January with the self-same models in sub- sequent months, the quality mix is kept constant in an attempt to avoid contaminating the price measurement through quality diﬀerences.
As will be seen later, how- ever, the resulting sample of models is one that gives less emphasis to models subsequently introduced which may have beneﬁted from more recent technological change and have diﬀerent price changes given the quality of services they provide.

One approach, which corrects for such quality changes but uses the whole sample, is that of the dummy variable hedonic regressions (see below).
Bode and van Dale´ n (2001), using a variety of formula- tions of hedonic regressions, found quality-corrected prices of these new automobiles to be about constant over this period, while their average nominal price increase was around 20 per cent.

7.21 It will be argued in Chapter 21 that observed changes in prices arise in theory from a number of sources, including quality changes, changes in tastes and prefer- ences, and changes in the technology of producers.
More formally, the observed data on prices are the locus of intersection of the demand curves of diﬀerent consumers with varying tastes and the supply curves of diﬀerent producers with possibly varying technologies of pro- duction.

The separation of the eﬀects of changes in tastes and preferences from quality changes is only possible in highly restrictive circumstances.

Chapter 8 suggests chaining or regular rebasing, so that weights – which reﬂect tastes and preferences – are not unduly out of date.
7.22 The changing mix of the observed character- istics of items is not the only concern.

There is also the practical problem of not always being able to observe or quantify characteristics such as the style, reliability, ease of use and safety of what is produced.

Chapter 16 of the System of National Accounts, 1993 (SNA 1993) on price and volume measurement notes factors other than changes in physical characteristics that give rise to improved quality.
These include ‘‘transporting a good to a location in which it is in greater demand is a process of production in its own right in which the good is trans- formed into a higher quality good’’.

The same good provided at a diﬀerent and more convenient location may command a higher price and be of a higher quality.
Furthermore, diﬀerent times of the day or periods of the year may also give rise to quality diﬀerences: ‘‘For example, electricity or transport provided at peak times must be treated as being of higher quality than the same amount of electricity or transport provided at oﬀ-peak times.

The fact that peaks exist shows that purchasers or users attach greater utility to the services at these times, while the marginal costs of production are usually higher at peak times .

.

.
.

.

.’’ Other diﬀerences, including the conditions of sale and circumstances or environment in which the goods or services are supplied or delivered, can make an important contribution to diﬀerences in quality.
A retailer, for example, may attract customers 102 by providing free delivery, credit opportunity or better variety, by being more accessible, by oﬀering shorter order times, smaller tailor-made orders, clearer label- ling, better support and advice, more convenient car parking or a wider range of brands, or simply by oper- ating in a more pleasant or fashionable environment.
These sorts of beneﬁts are not always speciﬁed in the item description because, ﬁrst, the services are provided without speciﬁc charge – they are incorporated in the prices of the goods sold.

Second, by matching the prices of models in speciﬁc outlets the level of such services is assumed to remain constant.

This does not mean, however, that conceptually such quality improvements should be outside the scope of the index.
If any such beneﬁts change, a price adjustment for the estimated value of the beneﬁts should be made.

7.23 To ask how to adjust prices for quality changes, it is ﬁrst necessary to ask what is meant by quality.

While there may be an intuition as to whether an item con- sumed in one period is better than its counterpart in the next, a theoretical framework will help in establishing the basis for such comparisons.

For example, an item of clothing is sampled and, after a few months, it is missing.
One option is to replace it with a similar item.

The nearest comparable option may have more cloth in it, or have a lining, be a diﬀerent colour, have diﬀerent but- tons, have better stitching or be considered to be better styled in some fashionable sense.

There is a need to put a price estimate on the diﬀerence in quality between the old and new items so that like can be compared with like.
To propose or criticize a quality adjustment procedure requires some concept of what is ideally required and how the procedure stands up to this.

Although such a discussion takes us away from the practicalities of the procedures for a while, its use will become apparent in subsequent sections.
A utility-based approach 7.24 In Chapter 17 a cost of living index (COLI) is deﬁned as the ratio of the minimum expenditures in the base and current period required to achieve a given standard of living or ‘‘utility’’.

Quality adjustments to prices involve trying to measure the price change for a product which has exhibited some change in its char- acteristics from an earlier period that provides a diﬀerent level of utility to the consumer.
The equating of the value of a quality change with the change in utility derived by the consumer, while falling naturally under a COLI framework, is not exclusive to it.

A cost of a ﬁxed basket of goods index (COGI) can also beneﬁt from regarding quality in this way.

While a COGI requires the pricing of a ﬁxed basket of products, some items will become unavailable and the replacement items selected to main- tain the sample may not be of the same quality.
The aim is to determine what proportion of the total price change results from a change in quality and what results from pure price change.

The concept of utility will be used to help with the former.

7.25 Note that the deﬁnition of a quality change is based on equating some change in characteristics to a diﬀerent level of utility provided.

Consider an example in which a new, improved quality item is substituted for an old one in period t, the consumer having to choose between the two.
Suppose that after the new quality item appeared, both qualities were oﬀered to a consumer at the same price, say pt=100.

The consumer was then asked to choose between them and naturally preferred the new quality.

Say the price of the old quality was then progressively reduced until it reached a point, say pt*= 75, at which the consumer was indiﬀerent as regards the choice between purchasing the old quality at pt*=75 and the new quality at pt=100.
The consumer might then select the old quality at 75 or the new one at 100.

Either way, the consumer would obtain the same utility, because of being indiﬀerent as to which to choose.

Any further decrease below pt*=75 would cause the con- sumer to switch back to the old quality.

7.26 The diﬀerence between pt and pt* would be a measure of the additional utility that the consumer placed on the new quality as compared with the old quality.
It would measure the maximum amount that the consumer was prepared to pay for the new quality over and above the price of the old quality.

In economic theory, as will be outlined in Chapter 21, if consumers (or households) are indiﬀerent between two purchases, the utility derived from them is the same.

The diﬀerence between 75 and 100 must therefore arise from the con- sumers’ valuation of the utility they derive from the two items: their quality diﬀerence.
The deﬁnition is sensible as a conceptual framework.

It naturally has problems relating to implementation, but this is not our concern here.

Our initial concern is with the provision of an analytical framework on which to ground our thinking and analysis.

7.27 The utility-based framework is concerned with the question of how consumers choose between items of diﬀerent qualities.
The answer, in part, is because more utility is derived from an item of higher quality than from an item of lower quality, and thus consumers prefer it.

But this does not explain why one item is bought rather than the other.

For this it is also necessary to know the relative price of one item with respect to the other, since if the lower-quality item is cheaper, it may still be pur- chased.
The above thought experiment to determine the price below which the old quality would be purchased, pt* 75, serves this purpose.

7.28 Deﬁning quality change in terms of its eﬀect on utility is of obvious beneﬁt to the economic approach to index numbers (Chapter 21).

Fixler and Zieschang (1992), Feenstra (1995), Triplett (1987) and Diewert (2003a) have developed theoretical frameworks for COLIs akin to those deﬁned in Chapter 21, but which also incorporate goods and services whose quality changes.
Silver and Heravi (2001a and 2003) and Kokoski et al.

(1999) have undertaken empirical studies based on these frameworks for comparisons over time and between geographical areas, respectively.

The use of utility as a guide towards understanding quality adjust- ments to prices is not, however, conﬁned to the economic theory of cost of living indices (Chapter 21).
Consumer price indices based on a ﬁxed basket concept have the pragmatic need to adjust for quality diﬀerences when an item is unavailable, and there is nothing in the deﬁnition ADJUSTING FOR QUALITY CHANGE of a ﬁxed basket index that precludes diﬀerences in utility being used as a guideline.

If item A is better than its old version, item B, it is because it delivers something more to the consumer who is willing to pay more.

That ‘‘thing’’ is called utility.


7.29 It is as well to distinguish between two concepts of value used in the analysis of quality adjustment: resource cost and user value.

The value users derive from their consumption is their utility.

Triplett (1990, pp.
Triplett (1990, pp.

222– 223) considers how a consumer price index diﬀers from a producer price index: Fisher and Shell (1972) were the ﬁrst to show that dif- ferent index number measurements (they considered out- put price indexes and consumer price indexes) imply alternative treatments of quality change, and that the theoretically appropriate treatments of quality change for these two indexes correspond respectively, to ‘‘resource- cost’’ and ‘‘user-value’’ measures.
Triplett (1983) derives this same result for cases where ‘‘quality change’’ is iden- tiﬁed with characteristics of goods – and therefore with empirical hedonic methods; the conclusions are that the resource cost of a characteristic is the appropriate quality adjustment for the output price index, and its user value is the quality adjustment for the COLI index or input index.

7.30 This position is not without diﬃculties.


Diewert (2002d) has advocated a user cost approach for the producer price output index.

This in part arises from the need to consolidate the inputs and outputs at constant prices in national accounts.

If diﬀerent quality adjust- ments are used for the same items in the producer price input index and the producer price output index, then the deﬂated constant price value added series, as their dif- ference, will not balance.
The issue arises generally in the ﬁeld of producer price indices, since it concerns the question of whether the producer price output index should use a user value concept.

It does not dispute the use of this concept in consumer price indices.

Conditional indices 7.31 The domain of a COGI is its ﬁxed basket of goods and services.

The use of a COLI framework requires consideration of wider issues concerning our quality of life.
There are changes in the social, physical and economic environment that require more or less expenditure to maintain a given level of utility.

Many factors aﬀect our welfare, and in practice not all can be included in a consumer price index.

It is thus appro- priate to consider indices that are conditional on exclu- ded factors remaining constant.

These generally include health status, the environment and the quantity and quality of government-provided goods and services.
The minimum expenditure necessary for achieving a given level of utility will increase as, for example, the police become less eﬀective.

Expenditure would then be nec- essary for better household security.

It would cost more to maintain a given level of utility than in the previous period.

Similarly, an outbreak of illness would lead to increased expenditure on medicines to maintain a given level of utility.

Bad winter weather increases heating bills to maintain the same utility as before.
In each case there is a very real sense in which the cost of living will have changed.

Yet it is not generally accepted that 103 CONSUMER PRICE INDEX MANUAL: THEORY AND PRACTICE the consumer price index should directly reﬂect such changes.

What should be reﬂected are changes in the prices of locks, medicines and fuel that arise because the demand for such items changes.
In addition, as more or less is spent on such items, the index should eventually incorporate such changes in the weighting as and when the weights are updated – and the more frequent the update, the better are such eﬀects incorporated.

But the index should not normally reﬂect short-run changes in the quantities used of security, medicine, heat and the like as a result of such external factors.
Gordon and Griliches (1997, p. 87) comment in a similar vein: It is not clear, moreover, whether events such as a colder winter, the appearance of AIDS, or a rise in the crime rate should be included in the deﬁnition of a price index.

A change in expenditures due to an unanticipated change in the weather should raise the price index only to the extent that energy prices go up, not quantities con- sumed.
If the event persists, ultimately it will aﬀect the commodity weights in the index, but that is a diﬀerent matter.

(Authors’ emphasis) 7.32 It may be inappropriate to disregard environ- mental factors if they seriously aﬀect a given group of people.

In such cases, indexing for special factors some- times takes place outside the index.

For example, a govern- ment may provide cold-weather payments to pensioners if the temperature falls below a threshold condition.
If a speciﬁc factor has a substantial eﬀect for a signiﬁcant index might be group of households, an additional compiled which includes the eﬀect.

An overview of methods of quality adjustment when matched items are unavailable 7.33 It is apparent from the above that quality adjustments to prices are not a simple matter of applying routine methods to prices in speciﬁed product areas.

A number of alternative approaches are suggested below.

Some will be more appropriate than others for speciﬁc product areas.
An understanding of the consumer mar- ket, technological features of the producing industry, and alternative data sources will all be required for the successful implementation of quality adjustments.

Spe- ciﬁc attention will need to be devoted to product areas with relatively high weights, where large proportions of items are turned over.

Some of the methods are not straightforward and require a level of expertise.
Quality adjustment needs to be implemented by developing a gradual approach on a product-by-product basis.

Such concerns should not be used as excuses for failing to attempt to estimate quality-adjusted prices.

The practice of statistical agencies in dealing with missing items, even if it is to ignore them, implicitly involves a quality adjust- ment.

Such an implicit approach may not be the most appropriate method, and may even be misleading.
The extent of quality changes and the pace of technological change require that appropriate methods be used.

7.34 To measure aggregate price changes, a repre- sentative sample of items is selected from a sample of outlets, along with a host of details that deﬁne each 104 price.

The items are repriced each month.

The detailed speciﬁcations are included on the repricing form each month as a prompt to help ensure that the same items are being priced.
Merkel (2000) has proposed that de- tailed checklists of item descriptions should be used, as any lack of clarity in the speciﬁcations may lead to errors.

It should be borne in mind that price collectors may have no incentive to report changes in speciﬁca- tions, since this will invariably involve additional work.
Attention should also be devoted to ensuring that the speciﬁcations used contain all pertinent, price-deter- mining elements, otherwise there may be cases in which the quality change would become invisible in the price measurement process.

7.35 When an item is missing in a month for reasons other than being oﬀ season or oﬀ cycle, the replacement may be of a diﬀerent quality – like may no longer be compared with like.
A number of approaches exist for dealing with such situations and are well documented for the consumer price index (CPI), as outlined in Turvey et al.

(1989), Moulton and Moses (1997), Armknecht et al.

(1997), Moulton et al.

(1999) and Triplett (2002).

Though the terminology diﬀers between authors and statistical agencies, they include: imputation – where no information is available to allow reasonable estimates to be made of the eﬀect on price of a quality change.
The price changes of all items, or of more or less similar items, are assumed to be the same as that for the missing item; overlap – used where no information is available to allow reasonable estimates to be made of the eﬀect on price of a quality change, but where a replacement item exists in the same period as the old item.
The price diﬀerence between the old item and its replace- ment in the overlap period is then used as a measure of the quality diﬀerence; direct comparison – if another item is directly com- parable, that is, it is so similar that it can be assumed to have had more or less the same quality character- istics as the missing one, its price replaces the una- vailable price.
Any diﬀerence in price level between the new and old is assumed to arise from price changes and not quality diﬀerences; explicit quality adjustment – where there is a sub- stantial diﬀerence between the quality of the old and replacement items, estimates of the eﬀect of quality diﬀerences on prices are made to enable quality- adjusted price comparisons to be made.

7.36 Before outlining and evaluating these methods it is as well to say something about the extent of the problem.
This arises when the item is unavailable.

It is not just a problem when comparable items are unavail- able, for the judgement as to what is and what is not comparable itself requires an estimate of quality diﬀer- ences.

Part of a statistical meta-information system for statistical oﬃces (outlined in Chapter 8) is to identify and monitor sectors that are prone to such replacements and whether the replacements used really are comparable.
Seminal studies in Canada and the United States throw some light on the extent of such replacements.

Moulton et al.

(1999) examined the extent to which items became unavailable for televisions in the compilation of the United States CPI.

Between 1993 and 1997, a total of 10,553 prices on televisions were used, of which 1,614 (15 per cent) were replacements, of which, in turn, 934 (57 per cent) were judged to be directly comparable.

Thus a typical television remained in the sample less than a year.
The Canadian experience for televisions over an almost identical period (1993 to November 1997) found 750 of the 10,050 prices (7.5 per cent) to be replacements.

Of these, 178 (24 per cent) were directly comparable, 162 (22 per cent) required a judgement and 410 (55 per cent) were ‘‘spliced’’ – the price diﬀerence between the replacement and the unavailable model in the two peri- ods being attributed to quality diﬀerences (Lowe, 1999).
Thus, there was wide variation in the frequency of total replacements, although the frequency of non- comparable replacements was roughly similar (6.4 per cent in the United States sample and 5.7 per cent in Canada).
Liegey (2000) found that of the 215 average (August 1999 to April 2000) monthly prices collected for major appliances for the United States CPI, 22 item replacements were required because of missing prices, of which comparable replacements were found for 16 and non-comparable replacements for the remaining six.

7.37 Information across a wider range of items is available for the United States.
Armknecht (1996) found that, over the three years 1993 to 1995, the annual average number of price observations collected for the United States CPI was 835,443, of which 59,385 (7.1 per cent) were substitutions (as opposed to imputations for missing values).

Of these substitutes, about half were carried out using comparable replacements, under a quarter using overall mean imputation, about 12 per cent using direct quality adjustment, and 10 per cent using class mean imputation.
It should be borne in mind that these ﬁgures ignore the implicit quality adjustments that take place when the Bureau of Labor Statistics rotates its sample between rebasing.

The overlap method is eﬀec- tively applied on sample rotation, the outlet and item samples being reselected for about one-ﬁfth of the geo- graphical areas, with prices of old and new items sampled in the same month.
All price-level diﬀerences between the old and new items are treated as quality diﬀerences as the new sample is spliced onto the old.

7.38 Methods of quality adjustment for prices are generally classiﬁed into implicit/imputed (or indirect) quality adjustment methods – the diﬀerences in termi- nology are notorious in this area – and explicit (or direct) methods.

Implicit and explicit methods are discussed below.


Both decompose the price change between the old item and its replacement into quality and pure price changes.

For explicit adjustments, however, an explicit estimate is made of the quality diﬀerence, usually on the basis of external information, and the pure price eﬀect is identiﬁed as a remainder.
For implicit adjustments, a measurement technique is used to compare the old item to the replacement item, in which the extent of the quality and pure price change is implicitly determined by the assumptions of the method.

The accuracy of the method relies on the veracity of the assumptions as opposed to the quality of the explicit estimate.
Explicit adjustments make use of separate estimates of the por- ADJUSTING FOR QUALITY CHANGE tion of prices ascribed to quality diﬀerences, so that the price of the original item can be compared with that of a replacement of the same quality.

The suitability of the explicit methods thus depends to a large extent on how good such estimates are, on average.
Implicit adjust- ments involve assumptions about price movements, and for these informed intuition or theory is relied upon – though in some cases national statistical oﬃces may make use of more speciﬁc empirical market knowledge.

Additive versus multiplicative adjustment 7.39 The quality adjustments to prices may be undertaken either by adding a ﬁxed amount or multi- plication by a ratio.
For example, where m is the old item and n its replacement for a comparison over periods t, t+1, t+2, the use of the overlap method in period t+1 requires the ratio pt+1 to be used as a measure of the relative quality diﬀerence between the old item and its replacement.
This ratio could then be multiplied by the price of the old item in period t, pt m, to obtain the quality- adjusted prices p*t n =pt+1 m as follows: m old item m replacement n t p t m t+1 pt+1 m pt+1 n t+2 pt+2 n 7.40 Such multiplicative formulations are generally advised, as the adjustment is invariant to the absolute value of the price.
It would otherwise be possible for the absolute value of the change in speciﬁcations to exceed the value of the item in some earlier or (with techno- logical advances) later period.

Yet there may be some items for which the worth of the constituent parts is not considered to be in proportion to the price.

In other words, the constituent parts have their own, intrinsic, absolute, additive worth, which remains constant over time.
Producers selling over the World Wide Web may, for example, include postage, which in some instances may remain the same irrespective of what is happening to price.

If postage is subsequently excluded from the price, this fall in quality should be valued as a ﬁxed sum.

Base versus current period adjustment 7.41 Two variants of the approaches to quality adjustment are to make the adjustment either to the price in the base period or to the price in the current period.
For example, in the overlap method, described above, the implicit quality adjustment coeﬃcient was used to adjust pt m. An alternative procedure would have been to multiply the ratio pt+1 by the price of the replace- to obtain the quality-adjusted price p*t+2 ment item pt+2 , etc.

The ﬁrst approach is easier since, once the base period price has been adjusted, no subsequent adjust- ments are required.

Each new replacement price can be compared with the adjusted base period price.
For multiplicative adjustments, the end result is the same whichever approach is used.

For additive adjustments, m =pt+1 n n n 105 CONSUMER PRICE INDEX MANUAL: THEORY AND PRACTICE the results diﬀer and it is more appropriate to make the adjustment to prices near to the overlap period.
Table 7.1 Example of adjustment (a) General illustration the implicit methods of quality Long-run versus short-run comparisons Outlet Item January February March April 7.42 Much of the analysis of quality adjustments in this manual has been undertaken by comparing prices between two periods, say, period 0 prices with those in a subsequent period 1.
For long-run comparisons the base period is taken as, say, period t and the index is compiled by comparing prices in period t ﬁrst with t+1; then t with t+2; then t with t+3, etc.

The short-run framework allows long-run comparisons, say, between periods t and t+3, to be built up as a sequence of links joined together by successive multiplication, say period t with t+2 and period t+2 with t+3; or with chaining, period t with t+1, t+1 with t+2 and t+2 with t+3.
The advantages of the short-run framework for imputations are dis- cussed in paragraphs 7.165 to 7.173.

7.43 Following a discussion of implicit and explicit methods of quality adjustment, issues relating to choice of method are considered.

The implicit and explicit adjustment methods are outlined under a standard long- run Laspeyres framework, in which prices in a base (or reference) period are compared with those in each subsequent period.
Where products are experiencing rapid technological change, however, these methods may be unsuitable.

The matching and repricing of like items, and ‘‘patching in’’ of quality-adjusted replace- ment prices when the matching fails, are appropriate when failures are the exception.

But in high-technology product markets likely to experience rapid turnover of models, they are the rule.

Alternative methods using chained or hedonic frameworks are therefore also con- sidered.
These are quite radical approaches to meet the needs of rapidly changing production portfolios.

Finally, the use of short-run comparisons as an alter- native to long-run ones is considered as an intermediary – and for imputation a more appropriate – approach.

Chapter 22 discusses issues relating to seasonal items in more detail.
Implicit methods of quality adjustment 7.44 This section discusses the following implicit methods of quality adjustment: overlap; overall mean or targeted mean imputation; class mean imputation; com- parable replacement; linked to show no price change; and carry-forward.

Overlap 7.45 Consider for illustration the case where the items are sampled in, say, January and prices are com- pared over the remaining months of the year.
Matched comparisons are undertaken between the January prices and their counterparts in successive months.

Five items are assumed to exist in January sold by two outlet types with prices p11; p21; p51; p61 and p81 (Table 7.1(a)).

At this the weights can be ignored level of aggregation, assuming only one quote is taken on each item.
A price 106 Specialized chain stores Department stores p11 p21 p51 p61 p81 p12 p22 p42 p52 p62 p82 p13 p33 p43 p53 p73 p83 p14 p34 p44 p54 p74 p84 (b) Numerical illustration Outlet Specialized chain stores Item 1 2 2. overlap – imputation – targeted imputation – comparable replacement 3 4 Department stores 5 6 – imputation – targeted imputation 7 8 January February March 4 5 10 12 5 6 7.5 11 12 10 10 6 6.9 6.56 7.2 6.5 6.5 8 12 13.13 12.533 14 10 index for February compared with January=100.0 is straightforward, in that prices of items 1, 2, 5, 6 and 8 only are used and compared by way of the geometric mean of price ratios, the Jevons index (which is equivalent to the ratio of the geometric mean in Feb- ruary over the geometric mean in January – see Chapter 20).
In March the prices for items 2 and 6 are missing, one from specialized chain stores and one from depart- ment stores.

7.46 Table 7.1(b) is a numerical counterpart to Table 7.1(a) to further illustrate the calculations.

The overlap method requires prices of the old and replacement items to be available in the same period.

In Table 7.1(a), in March item 2 has no price quote.

Its new replacement is, say, item 4.


The overlap method simply measures the ratio, in a common overlap period (February), of the prices of the old and replacement items (items 2 and 4, respectively).

This is taken to be an indicator of their quality diﬀerences.

The two approaches outlined above are apparent: either to insert a quality-adjusted price in January for item 4 and continue to use the replacement item 4 series, or to continue the item 2 series by patching in quality-adjusted item 4 prices.
Both yield the same answer.

Consider the former.

For a Jevons geometric mean for January to March for specialized chain stores only, assuming equal weights of unity: PJ( p1, p3)=[ p13=p11 p43=(( p42=p22) p21Þ]1=2 =[6=4 8=((7:5=6) 5)]1=2=1:386: (7.1) Note that the comparisons are long-run ones.

That is, they are between January and the month in question.
The short-run modiﬁed Laspeyres framework provides a basis for short-run changes based on data in each current month and the immediately preceding one.

In Tables 7.1(a) and (b) the comparison for specialized chain stores only would ﬁrst be undertaken between January and February using items 1 and 2, and this would be multi- plied by the comparison between February and March using items 1 and 4.
This still implicitly uses the diﬀer- ences in prices in the overlap in February between items 2 and 4 as a measure of this quality diﬀerence.

It yields the same result as before: 5 4 1 2 6 5 6 5 8 7:5 1 2 =1:386 The advantage of recording price changes for, say, Jan- uary to October in terms of January to September, and September to October, is that it allows the compiler to compare immediate month-on-month price changes for data editing purposes.
Moreover, it has quite speciﬁc advantages for the use of imputations (as discussed in paragraphs 7.53 to 7.68 below) for which diﬀerent results arise for the long- and short-run methods.

The long-run and short-run frameworks are discussed more fully in paragraphs 7.159 to 7.173.

7.47 The method is only as good as the validity of its underlying assumptions.

Consider i=1 .

.

.


m items where pt m is the price of item m in period t, pt+1 is the price of a replacement item n in period t+1, and there are overlap prices for both items in period t. Now item n replaces m, but is of a diﬀerent quality.

So let A(z) be the quality adjustment to pt+1 m such that the quality-adjusted price p*t+1 .
Very simply, the index for the item in question over the period t 1 to t+1 is: n which equates its quality to pt+1 m =A(zt+1)pt+1 n n n =pt n)=I t1,t+1 ( pt = m=pt1 pt+1 n pt1 m m ) ( pt+1 pt m pt n (7.2) 7.48 Now the quality adjustment to prices in period t+1 is deﬁned as previously, p*t+1 , which is the adjustment to pn in period t+1 which equates its utility to pm in period t+1 (had it existed then).
A desired measure of price changes between periods t1 and t+1 is thus: m =A(zt+1)pt+1 n ( p*t+1 m =pt1 m ) (7.3) The overlap formulation equals this when: p*t+1 pt1 m =A(zt+1) pt+1 n pt1 m = pt+1 n pt n pt m pt1 m A(zt+1)= pt m pt n and similarly for future periods of the series A(zt+1)= pt m pt n for p*t+i m pt1 m for i=2; .

.

.
.

.

T (7.4) ADJUSTING FOR QUALITY CHANGE The assumption is that the quality diﬀerence in any period equates to the price diﬀerence at the time of the splice.

The timing of the switch from m to n is thus crucial.

Unfortunately, price collectors usually hang onto an item so that the switch may take place at an unusual period of pricing, near the end of item m’s life cycle and the start of item n’s life cycle.

7.49 But what if the assumption does not hold?


What if the relative prices in period t, Rt=pt n, do not equal A(z) in some future period, say A(zt+i)=aiRt?

If ai=a, the comparisons of prices between future successive periods, say between t+3 and t+4, are unaﬀected, as would be expected, since item n is eﬀectively being compared with itself, m=pt p*t+4 m pt1 m .
p*t+3 m pt1 m = aRt aRt pt+4 n pt+3 n = pt+4 n pt+3 n (7.5) However, if diﬀerences in the relative prices of the old and replacement items vary over time, then: p*t+4 m pt1 m .

p*t+3 m pt1 m = a4 a3 pt+4 n pt+3 n (7.6) Note that the quality diﬀerence here is not related to the technical speciﬁcations or resource costs, but to the relative prices consumers pay.

7.50 Relative prices may also reﬂect unusual pricing policies aimed at minority segments of the market.
In the example of pharmaceutical drugs (Berndt et al., 2003), the overlap in prices of a generic with a branded product was argued to reﬂect the needs of two diﬀerent market segments.

The overlap method can be used with a judicious choice of the overlap period.

It should if possible be a period before the use of the replace- ment since in such periods the pricing may reﬂect a strategy to dump the old model to make way for the new one.
7.51 The overlap method is implicitly employed when samples of items are rotated.

That is, the old sample of items is used to compute the category index price change between periods t1 and t, and the new sample is used between t and t+1.

The ‘‘splicing’’ together of these index movements is justiﬁed by the assumption that – on a group-to-group rather than item-to-item level – diﬀerences in price levels at a common point in time accurately reﬂect diﬀerences in qualities.
7.52 The overlap method has at its roots a basis in the law of one price: that when a price diﬀerence is observed it must arise from some diﬀerence in physical quality or some such factors for which consumers are willing to pay a premium, such as the timing of the sale, location, convenience or conditions.

Economic theory would dictate that such price diﬀerences would not per- sist, given markets made up of rational producers and consumers.
However, Chapter 16 of SNA 1993 notes three reasons why this might fail: First, purchasers may not be properly informed about existing price diﬀerences and may therefore inadvertently buy at higher prices.

While they may be expected to search out for the lowest prices, costs are incurred in the process.
107 CONSUMER PRICE INDEX MANUAL: THEORY AND PRACTICE Secondly, purchasers may not be free to choose the price at which they purchase because the seller may be in a position to charge diﬀerent prices to diﬀerent categories of purchasers for identical goods and services sold under exactly the same circumstances – in other words, to practise price discrimination.

Thirdly, buyers may be unable to buy as much as they would like at a lower price because there is insuﬃcient supply available at that price.
This situation typically occurs when there are two parallel markets.

There may be a primary, or oﬃcial, market in which the quantities sold, and the prices at which they are sold, are subject to government or oﬃcial control, while there may be a secondary market – a free market or unoﬃcial market – whose existence may or may not be recognized oﬃcially.

Overall mean or targeted mean imputation It 7.53 This method uses the price changes of other items as estimates of the price changes of the missing items.
Consider a Jevons elementary price index, i.e., a geometric mean of price relatives (Chapter 20).

The prices of the missing items in the current period, say t+1, are imputed by multiplying their prices in the immediately preceding period t by the geometric mean of the price relatives of the remaining matched items between these two periods.

The comparison is then linked by multiplication to the price changes for previous periods.
the computationally most straightforward of methods since the estimate can be undertaken by simply dropping the items that are missing from both periods from the calculation.

In practice, the series is continued by including in the database the imputed prices.

is based on the assumption of similar price movements.
A targeted form of the method would use similar price move- ments of a cell or elementary aggregate of similar items, or be based on price changes at a higher level of aggregation if either the lower level had an insuﬃcient sample size or price changes at the higher level were judged to be more representative of the price changes of the missing item.

It is 7.54 In the example in Table 7.1, the January to February comparison for both outlet types is based on items 1, 2, 5, 6 and 8.
For March compared with Jan- uary – weights all equal to unity – the item 2 and item 6 prices are imputed using the short-run price change for February ( p2) compared with March ( p3) based on items 1, 5 and 8.

Since diﬀerent formulae are used for elementary aggregation, the calculations for the main three formulae are illustrated here (but see Chapter 20 on choice of formulae).
The geometric mean of the price ratios – the Jevons index – is: The ratio of the average (mean) prices – the Dutot index – is: PD( p2, p3)= P N .
P N ð7:8Þ p2 i =N p3 i =N i=1 i=1 =[( p13+p53+p83)=3 ( p12+p52+p82)=3] =(6+12+10)=(5+11+10)=1:0769, or a 7:69 per cent increase: The average (mean) of the price ratios – the Carli index – is: PC( p3, p2)= P N i=1 ( p3 i =p2 i )=N ð7:9Þ =[( p13=p12)+( p53=p52)+( p83=p82)]=3 =[6=5+12=11+10=10]=3=1:09697, or a 9:697 per cent increase: In practice, the imputed ﬁgure would be entered on the data sheet.
In Table 7.1(b) the overall mean imputations in March for items 2 and 6, using the Jevons index, are 1:0939 6=6:563 and 1:0939 12=13:127, respec- tively: these are shown in bold.

It should be noted that the Dutot index is in this instance lower that the Jevons index, a result not expected from the relationship established in Chapter 20.

The relationship in Chapter 20 assumed that the variance of prices would increase over time, while in Table 7.1(b) it decreases for the three items.
The arithmetic mean of relatives, the Carli index, equally weights each price change while the ratio of arithmetic means, the Dutot index, weights price changes according to the prices of the item in the base period relative to the sum of the base period prices.

Item 1 has a relatively low price (4), and thus weight, in the base period, but this item has the highest price increase (6/5).

The Dutot index is thus lower than the Carli index.
7.55 As noted above, it is also possible to reﬁne the imputation method by ‘‘targeting’’ the imputation: by including the weight for the unavailable items in group- ings likely to experience similar price changes, say by outlet type, speciﬁc product area or geographical region.

Any stratiﬁcation system used in the selection of outlets would facilitate this.
For example, in Table 7.1 assume that the price change of the missing item 2 in March is more likely to follow price changes of item 1 in specialized chain stores, and item 6 is more likely to experience similar price changes to those of items 5 and 8 in department stores.
For March compared with February, and weights all equal to unity, the geometric mean of the price ratios – the Jevons index – is: PJ( p2, p3)= Q N i=1 ( p3 i =p2 i )1=N (7.10) PJ( p2, p3)= Q N i=1 [ p3 i =p2 i ]1=N ð7:7Þ =[( p13=p12)2 ( p53=p52 p83=p82)3=2]1=5 =[(6=5)2 (12=11 10=10)3=2]1=5=1:1041: =[( p13=p12) ( p53=p52) ( p83=p82)]1=3 =[(6=5) (12=11) (10=10)]1=3=1:0939, or a 9:39 per cent increase: Note the weights used: for specialized chain stores the one price represents two prices, while for department stores the two prices represent three prices, or 3/2=1.5 each.
108 PC( p2, p3)= P N ( p3 i =p2 i ) .

N ð7:12Þ Q= 1 m x P i=1 p*t+1 m pt m x m(m x) mx P i=1 pt+1 i pt i The ratio of the average (mean) prices – the Dutot index – is: PD( p2, p3)= P N i=1 .

N p3 i .

P N i=1 .
N p2 i ð7:11Þ =[(2 p13+1:5 p53+1:5 p83)=5 (2 p12+1:5 p52+1:5 p82)=5] =[(2 6+1:5 12+1:5 10) (2 5+1:5 11+1:5 10)]=1:0843 The average (mean) of the price ratios – the Carli index – is: i=1 2 5 2 5 = = ( p13=p12)+ 3 5 [( p53=p52+p83=p82)=2] (6=5)+ 3 5 [(12=11+10=10)=2]=1:1073 7.56 Alternatively, and more simply, imputed ﬁg- ures could be entered in Table 7.1(b) for items 2 and 6 in March, just using specialized chain stores and depart- ment store price movements for items 2 and 6 respec- tively, and indices calculated accordingly.
Using a Jevons index, for item 2 the imputed value in March would be 6=5 6=7:2 and for item 6 it would be [(12=11) (10=10)]1=2=12:533.

It is thus apparent that not only does the choice of formula matter, as discussed in Chapter 20, but so too may the targeting of the impu- tation.

In practice, the sample of items in a targeted subgroup may be too small.
An appropriate stratum is required with a suﬃciently large sample size, but there may be a trade-oﬀ between the eﬃciency gains from the larger sample and the representativity of price changes achieved by that sample.

Stratiﬁcation by pro- duct area and region may be preferred to stratiﬁcation just by product area, if regional diﬀerences in price changes are expected, but the resulting sample size may be too small.
In general, the stratum used for the target should be based on the analyst’s knowledge of the market, as well as an understanding of similarities of price changes between and within strata, and the relia- bility of the sample available to be representative of price changes.

n 7.57 The underlying assumptions of these methods require some analysis since, as discussed by Triplett (1999 they are often misunderstood.

Consider and 2002), i=1 .

.

.


m items where, as before, pt m is the price of item m in period t, pt+1 is the price of a replacement item n in period t+1.

Now n replaces m, but is of a diﬀerent quality.

So, as before, let A(z) be the quality adjustment n which equates its quality services or utility to pt+1 to pt+1 m such that the quality-adjusted price p*t+1 .

For the imputation method to work, the average price chan- ges of the i=1 .

.

.
.

.

m items, including the quality-adjusted price p*t+1 , given on the left-hand side of equation (7.13), must equal the average price change from just using the overall mean of the rest of the i=1 .

.

.

m 1 items, on the m =A(z)pt+1 m n ADJUSTING FOR QUALITY CHANGE right-hand side of equation (7.13).

The discrepancy or bias from the method is the balancing term Q.

It is the implicit adjustment that allows the method to work.
The arithmetic formulation is given here, though a similar geometric one can be readily formulated.
The equation for one unavailable item is given by: 1 m p*t+1 m pt m m1 + P i=1 pt+1 i pt i = 1 (m 1) m1 P i=1 pt+1 i pt i +Q (7.13) Q= 1 m p*t+1 m pt m 1 m(m 1) m1 P i=1 pt+1 i pt i and for x unavailable items by: (7.14) (7.15) 7.58 The relationships are readily visualized if r1 is deﬁned as the arithmetic mean of price changes of items that continue to be recorded and r2 of quality-adjusted unavailable items.
For the arithmetic case, where mx r1= P pt+1 i =pt i (m x) and i=1 x r2= P i=1 =pt i x (7.16) p*t+1 i then the ratio of arithmetic mean biases from sub- stituting equation (7.16) in (7.15) is: Q= x m (r2 r1) (7.17) which equals zero when r1=r2.

The bias depends on the ratio of unavailable values and the diﬀerence between the mean of price changes for existing items and the mean of quality-adjusted replacement price changes.
The bias decreases as either (x=m) or the diﬀerence between r1 and r2 decreases.

Furthermore, the method is reliant on a comparison between price changes for existing items and quality-adjusted price changes for the replacement or unavailable comparison.

This is more likely to be justi- ﬁed than a comparison without the quality adjustment to prices.
For example, suppose there were m=3 items, each with a price of 100 in period t. Let the t+1 prices be 120 for two items, but assume the third is unavailable, i.e., x=1 and is replaced by an item with a price of 140, of which 20 is attributable to quality diﬀerences.
Then the arithmetic bias as given in equations (7.16) and (7.17), where x=1 and m=3, is 1 3 ( 20+140)=100 120 100 + 120 100 =2 =0 Had the bias depended on the unadjusted price of 140 compared with 100, the imputation would be prone to serious error.
In this calculation, the direction of the bias is given by (r2 r1) and does not depend on whether in other words quality is improving or deteriorating, whether A(z) > pt+1 or A(z) < pt+1 , a quality improvement, it is still possible that r2 < r1 and for the bias to be negative, a point stressed by Triplett (2002).

.

If A(z) > pt+1 n n n 109 CONSUMER PRICE INDEX MANUAL: THEORY AND PRACTICE 7.59 The analysis here is framed in terms of a short- run price change framework.
That is, the short-run price changes between the prices in a period and those in the preceding period are used for the imputation.

This is diﬀerent from the long-run imputation where a base period price is compared with prices in subsequent months, and where the implicit assumptions are more restrictive.
7.60 Table 7.2 provides an illustration in which the (mean) price change of items that continue to exist, r1, is allowed to vary for values between 1.00 and 1.5 – corresponding to a variation between no price change and a 50 per cent increase.

The (mean) price change of the quality-adjusted new items compared with the items they are replacing is assumed not to change, i.e., r2= 1.00.
The bias is given for ratios of missing values of 0.01, 0.05, 0.1, 0.25 and 0.5, both for arithmetic means and geometric means.

For example, if 50 per cent of price quotes are missing and the missing quality-adjus- ted prices do not change, but the prices of existing items increase by 5 per cent (r1=1.05), then the bias for the geometric mean is represented by the proportional fac- tor 0.9759; i.e., instead of 1.05, the index should be 0.9759 1.05=1.0247.
For an arithmetic mean, the bias is 0.025; instead of 1.05 it should be 1.025.

7.61 Equation (7.17) shows that the ratio x/m and the diﬀerence between r1 and r2 determine the bias.

Table 7.2 shows that the bias can be quite substantial when x/m is relatively large.
For example, for x/m=0.25, an inﬂa- tion rate of 5 per cent for existing items translates to an index change of 3.73 per cent and 3.75 per cent for the geometric and arithmetic formulations, respec- tively, when r2=1.00, i.e., when quality-adjusted prices of unavailable items are constant.

Instead of being 1.0373 or 1.0375, ignoring the unavailable items would give a result of 1.05.
Even with 10 per cent missing (x/ m=0.1), an inﬂation rate of 5 per cent for existing items translates to 4.45 per cent and 4.5 per cent for the geo- metric and arithmetic formulations, respectively, when r2=1.00.

Considering a fairly low ratio of x/m, say 0.05, then even when r2=1.00 and r1=1.20, Table 7.2 shows that the corrected rates of inﬂation should be 18.9 per cent and 19 per cent for the geometric and arithmetic formulations, respectively.
In competitive markets, r1 and r2 are unlikely to diﬀer by substantial amounts since r2 is a price comparison between the new item and the old item after adjusting for quality diﬀerences.

If r1 and r2 are the same, then there would be no bias from the method even if x/m=0.9.

There may, however, be more sampling error.

It should be borne in mind that it is not appropriate to compare bias between the arithmetic and geometric means, at least in the form they take in Table 7.2.
The latter would have a lower mean, rendering comparisons of bias meaningless.

7.62 An awareness of the market conditions relating to the commodities concerned is instructive in under- standing likely diﬀerences between r1 and r2.

The con- cern here is when prices vary over the life cycle of the items.

Thus, for example, at the introduction of a new model, the price change may be quite diﬀerent from price changes of other existing items.
Thus assumptions of similar price changes, even with quality adjustment, might be inappropriate.

Greenlees (2000) gives the exam- ple of personal computers: new computers enter the mar- ket at prices equal to, or lower than, prices of previous models, but with greater speed and capability.

An as- sumption that r1=r2 could not be justiﬁed.


He continues with the example of apparel, in which new clothing enters the market at relatively high quality-adjusted prices, while old, end-of-season or out-of-style clothes are being discounted.

Again there will be bias, as r1 diﬀers from r2.

7.63 Some of these diﬀerences arise because markets are composed of diﬀerent segments of consumers.
Indeed, the very training of consumer marketers involves consideration of developing diﬀerent market segments and ascribing to each appropriate pricing, product quality, promotion and place (method of distribution) – the 4Ps of the marketing mix (Kotler, 1991).

In addition, consumer marketers are taught to plan the marketing mix for the life cycle of items.

Such planning allows for diﬀerent inputs of each of these marketing mix variables at diﬀerent points in the life cycle.
This includes ‘‘price skimming’’ during the period of introduction, when Table 7.2 Example of the bias from implicit quality adjustment when the (mean) price change of quality-adjusted new items compared with the items they are replacing is assumed not to change (r2=1.00) Geometric mean Arithmetic mean Ratio of missing items, x/m 0.01 0.05 0.1 0.25 0.5 Ratio of missing items, x/m 0.01 0.05 0.1 0.25 0.5 r1 0.015
0.125 0 0.005 0.01 0.015 0.02 0.025 0.05 0.075 0.1 0.15 0.25 r1=(mean) price change for items that continue to exist.
110 higher prices are charged to skim oﬀ the surplus from segments of consumers willing to pay more.

The eco- nomic theory of price discrimination would also predict such behaviour.

Thus the quality-adjusted price change of an old item compared with a new replacement item may be higher than price changes of other items in the product group.

After the introduction of the new item its prices may fall relative to others in the group.
There may be no law of one price change for diﬀerentiated items within a market.

Berndt et al.

(2003) clearly show how, after patents expire, the price of branded prescription pharmaceuticals can increase with the entry of new generic pharmaceuticals at a lower price, as particularly loyal, less price-sensitive customers maintain their al- legiance to the branded pharmaceuticals.
7.64 There is thus little in economic or marketing theory to support any expectation of similar (quality- adjusted) price changes for new and replacement items, as compared to other items in the product group.

Some knowledge of the realities of the particular market under study would be helpful when considering the suitability of this approach.

Two aspects need to be considered in any decision to use the imputation approach.

The ﬁrst is the proportion of replacements; Table 7.2 provides guidance here.
The second is the expected diﬀerence between r1 and r2.

It is clear from the above discussion that there are markets in which they are unlikely to be similar.

This is not to say the method should not be used.

It is a simple and expedient approach.

What arguably should not happen is that it is used by default, without any prior evaluation of expected price changes and the timing of the switch.

Furthermore, its use should be targeted, by selecting items expected to have similar price changes.
The selection of such items, however, should take account of the need to include a suﬃciently large sample so that the estimate is not subject to undue sampling error.

7.65 The manner in which these calculations are undertaken is also worth considering.

In its simplest form, the pro forma setting for the calculations, say on a spreadsheet, would usually have each item description and its prices recorded on a monthly basis.
The imputed prices of the missing items are inserted into the spread- sheet, and are highlighted to show that they are imputed.

The need to highlight such prices is, ﬁrst, because they should not be used in subsequent imputations as if they were actual prices.

Second, the inclusion of imputed values may give a false impression of a larger sample size than actually exists.
Care should be taken in any audit of the number of prices used in the compilation of the index to code such observations as ‘‘imputed’’.

7.66 The method described above is an illustration of a short-run imputation.

As is discussed in paragraphs 7.165 to 7.173 below, there is a strong case for using short-run imputations as against long-run ones.
Class mean imputation class mean (or 7.67 The substitution relative) method of implicit quality adjustment to prices as used in the United States CPI is discussed by Schultz (1996), Reinsdorf, Liegey and Stewart (1996), Armknecht, Lane ADJUSTING FOR QUALITY CHANGE and Stewart (1997), and Armknecht and Maitland-Smith (1999).
It arose from concerns similar to those con- sidered in the previous section, that unusual price changes were found in the early introductory period, when new models were being introduced, particularly for consumer durables.
Moulton and Moses (1997), using United States CPI data for 1995 in their study of selected products, found the average pure price change to be only 0.12 per cent for identical items being repriced (on a monthly or bimonthly basis), compared to an average 2.51 per cent for comparable substitutes – items judged equivalent to the items they replaced.

The corresponding average price change for directly substituted quality- adjusted price changes was 2.66 per cent.
Thus, the price movement of continuing items appears to be a ﬂawed proxy for the pure price component of the diﬀerence between old and replacement items.

7.68 The class mean method was adopted in the United States CPI for cars in 1989 and was phased in for most other non-food commodities, beginning in 1992.

It diﬀered from the overall mean imputation method only in the source for the imputed rate of price change for the old item in period t+1.
Rather than using the category index change, obtained using all the non-missing items in the category, the imputed rate of price change was based on constant quality replacement items – those items that were judged comparable or that were quality-adjusted directly.
The class mean approach was seen as an improve- ment on the overall mean imputation approach because the imputed price changes were based on items that had not just been replaced, but whose replacement price had beneﬁted from a quality adjustment or the new replace- ment item had been judged to be directly comparable.

It may be the case, however, that suﬃciently large samples of comparable substitutes or directly quality-adjusted items are unavailable.
Or it may be that the quality adjustments and selection of comparable items are not deemed suﬃciently reliable.

In that case, a targeted imputation might be considered.

The targeted mean is less ambitious in that it seeks only to capture price changes of similar items, irrespective of their point in the life cycle.

Yet it is an improvement on the overall mean imputation, as long as suﬃciently large sample sizes are used.
Comparable replacement 7.69 The comparable replacement method requires the respondent to make a judgement that the replace- ment is of a similar quality to the old item and any price changes are untainted by quality changes.

For specia- lized chain stores in Table 7.1(b), item 3 might be judged to be comparable to item 2 and its prices in subsequent months might be used to continue the series.
The price of item 3 (6.5) in March would be used as the price in March of item 2, whose January to March price change would be 6.5/6 100=1.0833 or 8.33 per cent.
Lowe (1999) notes the common practice of manufacturers of televi- sion sets to change model numbers with a new produc- tion run, though nothing physically has changed, or when small changes take place in speciﬁcations, such as the type of remote controls, or the number or placement 111 CONSUMER PRICE INDEX MANUAL: THEORY AND PRACTICE of jacks.
The method of comparable replacement relies on the eﬃcacy of the price collectors and, in turn, on the adequacy of the speciﬁcations used as a description of the items.

Statistical agencies may rightly be wary of sample sizes being reduced by dropping items for which prices need to be imputed, and also wary of the intensive use of resources to make explicit estimates as outlined below.

The use of repriced items of a comparable speciﬁcation has much to commend it.
If the quality of items is improving, however, the preceding item will be inferior to the current one.

Continually ignoring small changes in the quality of replacements can lead to an upward bias in the index.

The extent of the problem will depend on the proportion of such occurrences, the extent to which comparable items are accepted as being so despite quality diﬀerences, and the weight attached to those items.
Proposals in Chapter 8 to monitor types of quality adjustment methods by product area provide a basis for a strategy for applying explicit adjustments where they are most needed.

Linked to show no price change 7.70 Linking attributes any price change between the replacement item in the current period and the old item in the preceding period to the change in quality.

For example, in Table 7.1(b), a replacement item 7 is selected from a department store for the missing March item 6.
Items 6 and 7 may be of diﬀerent quality, the price dif- ference being quite large.

The change in price is assumed to be attributable to a change in quality.

An estimate is made for p72 by equating it to p73, to show no change, i.e., the assumed price of item 7 in February is 14 in Table 7.1(b).

There is thus assumed to be no price change over the period February to March for item 7.

The for item 6 is (12=12) January to March result (14=14)=1:00, indicating no change.
For the period March to April, however, the price of item 7 in March can be compared with the imputed p72 for February and linked to the preceding results.

So the January to April comparison is composed of the January to February comparison for item 6, linked to (multiplied by) the February to April comparison for item 7.

This linking is analogous to the procedures used for the chained and short-run framework discussed in paragraphs 7.153 to 7.158 and 7.171 to 7.173 below.
The method is born out of circumstances where comparable replacement items are not available and there are relatively large price dif- ferences between the old and replacement items, these being from diﬀerent price bases and of diﬀerent qualities.

It is not possible to separate out how much of this dif- ference is attributable to price changes and how much to quality changes, so the method attributes it all to quality and holds price constant.
The method introduces a degree of undue price stability into the index.

It may well be the case that the period of replacement is when sub- stantial price changes are taking place and that these are wrongly attributed to quality changes by this method.

Article 5 of the European Commission (EC) Regulation No.

1749/96 requires Member States to avoid ‘‘auto- to the matic linking’’.
Such linking is equivalent assumption that the diﬀerence in price between two 112 successive models is wholly attributable to a diﬀerence in quality (Eurostat, 2001a, p. 125).

Carry-forward 7.71 With the carry-forward method, when an item becomes unavailable, say in period t, the price change calculation uses the old t 1 price, simply carried for- ward as if there were no change.
Thus from Table 7.1(a) for specialized chain stores for the period January to March, the Jevons and Dutot indices (Chapter 20) are: PJ( p1, p3)=[( p13=p11 p22=p21)]1=2 PD( p1, p3)=[( p13+p22)=( p11+p21)] and (7.18) with p22 ﬁlling in for the missing p23.

This introduces undue stability into the index, which is aggravated if the old price, p22, continues to be used to ﬁll in the unob- served prices in subsequent periods.
It induces an inap- propriate amount of stability into the index and may give a misleading impression of the active sample size.

The practice of the carry-forward method is banned under Article (6) of the EC Regulation No.

1749/96 for Har- monized Indices of Consumer Prices (Eurostat, 2001a, p. 126).

To use this method an assumption is made that the price from this outlet would not change.

This method should only be used if it is fairly certain that there would be no price change.
Explicit methods of quality adjustment 7.72 The aforementioned methods do not rely on explicit information on the value of the change in quality, A(z).

This section discusses the following methods that rely on obtaining an explicit valuation of the quality diﬀerence: expert judgement; quantity adjustment; dif- ferences in production or option costs; and the hedonic approach.
Expert judgement 7.73 Hoven (1999) describes comparable replace- ment as a special case of subjective quality adjustment, because the determination of product equivalence is based on the judgement of the commodity specialist.

One objection to subjective methods is the inability to provide results that can be independently replicated.

Yet in com- parable replacement, and for the selection of repre- sentative items, a subjective element is part of normal procedure.
This is not, of course, an argument for expand- ing the use of subjective methods.

7.74 Hoﬀman (1999) describes a possibly unique alternative for quality adjustment of replacement items in the German CPI.

When a new product is more expensive than the item it replaces, a ﬂexible adjustment factor can be employed, attributing none, some, or all of the price diﬀerence to improved quality.
In particular, when no precise information is available on which to make a quality determination, it is permissible for an adjustment to be made of 50 per cent of the price dif- ference.

The guidelines used in Germany since 1997 replaced ﬂawed procedures in which the particular individual quality adjustments methods chosen for depended on the diﬀerence in price alone.

As Hoﬀmann notes, however, even in the current approach no quality adjustment is made if the new item is less expensive than the old.
Consequently, problems could arise if an increase in quality were accompanied by a decrease in price (or vice versa).

The methods used in the German CPI are needed because quality adjustments for most goods are made not in the central CPI oﬃce but by price collectors in the ﬁeld.

Wide use of the hedonic and pro- duction cost approaches is precluded under these condi- tions.
Thus, the organizational structure of the statistical agency, as well as its funding level, will necessarily inﬂu- ence its choice of quality adjustment methods.

7.75 Reports by consumer associations and product evaluations in consumer magazines are not advised by Turvey (1998), who cites a study which correlated qual- ity ratings and prices for 135 products categories using Consumer Reports.
The average correlation was 0.26, with over half having a positive association, just over a third no association and the rest a negative one.

He also argues against ‘‘best buy’’ estimates, which are expert views as to what a sensible consumer should pay, as opposed to what the market price will be (see also Combris, Lecocqs and Visser, 1997).

7.76 The use of expert views as to consumer calcu- lations may be appropriate for highly complex items feasible.
Experts where alternative methods are not should be guided with regard to the nature of the estimate required.

More than one expert should be chosen and, where possible, the experts should be from diﬀerent backgrounds.

It is also advisable to give the experts some indication of the interval in which their estimate should lie.

The well-known Delphi method (for example, see Czinkota and Ronkainen, 1997) may be applicable.
In this approach a panel of experts never meet, to avoid any ‘‘bandwagon’’ eﬀect regarding their estimates.

They are asked to provide an estimate of the average response and the range of likely responses.

The median is taken of these estimates and any estimate that is considered extreme is sent back to the expert concerned, who is asked to account for possible reasons behind the diﬀerence.
It may be that the particular expert has a useful perspective on the problem, which the other experts had not considered.

If the expert argues the case convincingly, the response is fed back to the panel who are asked if they wish to change their views.

A new median is taken, and further iterations are possible.

The Delphi method is time-consuming and expensive, but it reﬂects the care needed in such matters.
If an adjustment is required for a product area with a large weighting in the CPI, and no other techniques are available, it is a possible alternative.

Quantity adjustment 7.77 Quantity adjustment is one of the most straight- forward explicit adjustments to undertake.

It is appli- cable when the site of the replacement item diﬀers from that of the available item.

In some situations there is a readily available quantity metric that can be used to compare the items.
Examples are the number of units in a ADJUSTING FOR QUALITY CHANGE package (e.g., paper plates or vitamin pills), the size or weight of a container (e.g., kilogram of ﬂour, litre of cooking oil), or the size of sheets or towels.

Quality adjustment to prices can be accomplished by scaling the price of the old or new item by the ratio of quanti- ties.
The index production system may do this scaling adjustment automatically, by converting all prices in the category to a price per unit of size, weight or number.

Scaling is important.

For example, if cooking oil is now sold in 5 litre containers instead of 2.5 litre ones, it should not be the case that prices have doubled.

7.78 There is, however, a second aspect.

In the phar- maceutical context, for example, prices of bottles of pills of diﬀerent sizes diﬀer.
A bottle of 100 pills, each having 50 milligrams of a drug, is not the same as a bottle of 50 pills of 100 milligrams, even though both bottles contain 5,000 milligrams of the same drug.

If there is a change, say, to a larger size container, and a unit price decrease of 2 per cent accompanies this change, then it should not be regarded as a price fall of 2 per cent if consumers gain less utility from the larger and more inconvenient containers.
In practice it will be diﬃcult to determine what propor- tion of the price fall is attributable to quality and what proportion to price.

A general policy is not to auto- matically interpret unit price changes arising from packaging size changes as pure price changes, if contrary information is available.

7.79 Consider a further example: a branded bag of ﬂour previously available in a 0.5 kilogram bag priced at 1.5 is replaced by a 0.75 kilogram bag priced at 2.25.
The main concern here is with rescaling the quantities.

The method would use the relative quantities of ﬂour in each bag for the adjustment.

The prices may have increased by [(2:25=1:5) 100=150] 50 per cent but the quality- adjusted prices (i.e.

prices adjusted by size) have rem- ained constant [(2:25=1:5) (0:5=0:75) 100=100]: The approach can be outlined in a more elaborate manner by recourse to Figure 7.1.
The concern here is with the part of the unbroken line between the (price, quantity) coor- dinates (1.5, 0.5) and (2.25, 0.75), both of which have unit prices of 3 (price=1.5/0.5 and 2.25/0.75).

There should be no change in quality-adjusted prices.

The symbol D denotes a change.


The slope of the line is b Figure 7.1 Quality adjustment for different-sized items 2.5 2 1.5 1 0.5 e c i r P 0 0 5 3 ∆price 3 ∆size β = ∆price/∆size 0.25 0.5 0.75 Size in kilograms 113 CONSUMER PRICE INDEX MANUAL: THEORY AND PRACTICE which is Dprice=Dsize=(2:25 1:5)=(0:75 0:50)=3, i.e., the change in price arising from a unit (kilogram) change in size.
The quality- (size-) adjusted price in period t 1 of the old m bag is: Table 7.3 Example of size, price and unit price of bags of ﬂour Size (kilograms) First price First unit price Second price Second unit price m =pt1 ^ppt1 m +bDsize=1:5+3(0:75 0:5)=2:25 (7.19) The quality-adjusted price change shows no change, as before: 0.25 0.5 0.75 0.75 1.50 2.25 3 3 3 1.25 1.50 2.25 5 3 3 n=^ppt1 pt m =2:25=2:25=1:00 The approach is outlined in this form so that it can be seen as a special case of the hedonic approach (discussed below), where price is related to a number of quality characteristics of which size may be only one.
that the unit 7.80 The method can be seen to be successful on intuitive grounds as long as the unit price of diﬀerent- sized bags remains constant.
If the switch was from the replacement of the 0.5 kilogram bag to a 0.25 kilogram one priced at 0.75, as shown by the continuation to coordinate (0.75, 0.25) of the unbroken line in Figure 7.1, the quality-adjusted prices would again not change, assuming, however, (kilogram) prices were 5, 3 and 3 for the 0.25, 0.5 and 0.75 kilogram bags, respectively, as shown in Table 7.3 and in Figure 7.1 (including the broken line).
Then the measure of quality- adjusted price change would depend on whether the 0.5 kilogram bag was replaced by the 0.25 kilogram one (a 67 per cent increase) or the 0.75 kilogram one (no change).

This is not satisfactory because the choice of replacement size is arbitrary.

The rationale behind the quality adjustment process is to ask: does the diﬀerence in unit price in each case reﬂect diﬀerent levels of utility?

If so, adjustments should be made to the unit prices to bring them into line.
If not, adjustments should be made to the unit prices for that proportion attributable to diﬀerences in utility gained from, say, more con- venient packaging or the availability of smaller lots.

It may be obvious from the nature of the product that an item packaged in a very small size with a dis- proportionately high unit price carries an unusually high proﬁt margin, and that an appropriate replacement for a large-sized item would not be this very small one.
Differences in production or option costs 7.81 A natural approach to quality adjustment is to adjust the price of an old item by an amount equal to the resource costs of the additional features of the new item; i.e., to compare relative prices using: n=^ppt1 pt m where m =pt1 ^ppt1 m +x (7.20) and x is the value of the additional features in period t 1 prices.

This value should be a consumer’s valuation, reﬂecting the additional ﬂow of services or utility.
One source of data is the manufacturers.

They would be asked to provide data on production costs, to which retail mark-ups and associated indirect taxes would be added.

This approach is most practicable in markets where there is a relatively small number of manu- facturers, and where updates of models are infrequent 114 and predictable.

It only works if there is good commu- nication between manufacturers and the statistical agency staﬀ.
It is particularly suitable when the quality adjustments are also being undertaken to calculate the producer price index (PPI) or other price programmes.

Greenlees (2000) provides an example for new trucks and motor vehicles in the United States in 1999.

Just prior to the introduction of the annual models, BLS staﬀ visit selected manufacturers to collect cost information.
The data are used in the PPI and International Comparison Programmes as well as in the CPI, and the information- gathering activity is a joint operation of the three pro- grammes.

Allowable product changes for the purpose of quality adjustments include occupant safety enhance- ments, mechanical and electrical improvements to over- all vehicle operation or eﬃciency, changes that aﬀect length of service or need for repair, and changes aﬀecting comfort or convenience.
7.82 Bearing in mind the caveat in paragraph 7.30, the producer orientation of the PPI implies that resource cost is the appropriate criterion for quality adjustment to prices (Triplett, 1983).

One distinction, then, between the use of producer cost estimates in the CPI and PPI is that only the former programme will add retail mark- ups and indirect taxes.

Another important diﬀerence may occur in situations where product improvements are mandated by government.
Some of these mandated improvements provide no direct beneﬁt to the pur- chaser.

In these cases it is appropriate to make a quality adjustment to prices for the associated resource cost in the PPI, but not in the CPI, where the appropriate cri- terion is user value.

If only production cost data are available, then estimates of the retail mark-up must take into account the (average) age of the models under consideration.

Mark-ups will decrease as models come to the end of their life cycles.
Therefore, mark-ups based on models at the end of their life cycle should not be applied to the production costs of models at the start of their life cycle.

7.83 Because of these diﬃculties in using the pro- duction cost approach, the option cost method is gen- erally preferred.

Often it is the retail price of an option that is available and this of course includes the mark-up for proﬁt.

Consider an example of the price of an option being used to adjust for quality.
Let the prices for an item in periods t 1 and t be 10,000 and 10,500, respectively, but assume the price in period t is for the item with a new feature or ‘‘option’’, and let the price of the additional feature in period t be known to be 300.

Then the price change would be 10,200/10,000=1.02 or 2.0 per cent.

The adjustment may take a multiplicative form (see paragraphs 7.39–7.40 above): the additional option is worth 300/10,500=0.028571 of the period t price.
The adjusted price in period t 1 is therefore 10,000 1:028571=10; 285:71 and the price change 10,500/ 10,285.71=1.020833 or about 2.08 per cent.

If in sub- sequent periods either of these elements changes, then so too must ^ppn;t1 for those comparisons.

The option cost method is thus a method for use in stable markets with stable technologies.
Alternatively, it may be preferable to estimate a one-oﬀ adjustment to the preceding base period price and then compare all subsequent prices with the new option to this estimate; i.e.

10,500/10,300= 1.019417 or approximately 2 per cent.

7.84 Option costs are thus useful in situations in which the old and new items diﬀer by quantiﬁable characteristics that can be valued in monetary terms by reference to market prices.
For example, nuts may be available roasted or unroasted, and food items may be available cooked or uncooked.

Consider the addition of a feature to a car model.

The feature may have been available as an option either in the prior period or currently for other models, providing an absolute or consumer valuation.
Armknecht and proportional Maitland-Smith (1999) note that when radial tyres became a standard feature on new cars, the price of adding optional radial tyres was used to determine the quality adjustments in the United States CPI.

The valuation of a quantiﬁable product feature may be readily available from the comparison of diﬀerent pro- duct prices.

Turvey et al.

(1989) give the example of whiskies of diﬀerent proofs (percentage alcohol content).
The quality adjustment for a change in the alcohol content of one product may be inferred from the market relationship between proof and price.

7.85 Consider the addition of a feature to a product – say an installed automatic ice-maker in a refrigerator (Shepler, 2000).

Refrigerators can be sold as standard or with an installed automatic ice-maker.
The price col- lector may always have collected prices on the standard model, but this may no longer be in production, being replaced by a model with an installed automatic ice- maker.

The cost of the option is thus known from before and a continuing series can be developed by using equation (7.20) and simply adjusting the old price in the base period for the option cost.

Even this process may have its problems.


First, the cost of producing some- thing as standard may be lower than when it was an option, all new refrigerators now having the installed automatic ice-maker.

This saving may be passed on, at least in part, to the consumer.

The option cost method would thus understate a price increase.

Triplett (2002) cites a study by Levy et al.

(1999) in which a car theft system was installed as standard but disabled when the option was not required.
It was seemingly cheaper to produce this way.

Second, by including something as standard the consumer’s valuation of the option may fall since buyers cannot refuse it.

Some consumers may attribute little value to the option.

The overall eﬀect would be that the estimate of the option cost, priced for those who choose it, is likely to be higher than the implicit average price consumers would pay for it as standard.
Estimates of the eﬀect on price of this dis- crepancy should in principle be made, though in practice are quite diﬃcult.

ADJUSTING FOR QUALITY CHANGE n=^ppt1 m where ^ppt1 7.86 Option cost adjustments can be seen to be similar to quantity adjustments, except that instead of size being the additional quality feature of the replace- ment, the added quality can be any other individual feature.
The comparison is: pt m =pt1 m + bDz for an individual z characteristic where Dz=(zt n zt1 m ).

The characteristics may be the size of the random access memory (RAM) of a personal computer (PC) when a speciﬁc model of PC is replaced by a model that is identical except for the amount of RAM it possesses.

If the relationship between price and RAM is linear, the above formulation is appropriate.
Many web pages give the price of additional RAM as being independent of other features of PCs, and a linear adjustment is appro- priate.

Bear in mind that a linear formulation values the worth of a ﬁxed additional amount of RAM to be the same, irrespective of the amount of RAM the machine possesses.

7.87 The relationship may, of course, be non-linear.

Say, for example, for every additional 1 per cent of x, y increases by 1.5 per cent (b=1:015).
In this case, m =pt1 ^ppt1 m bz (7.21) n=^ppt1 m for pt as a measure of quality-adjusted price changes.

Again the z change may reﬂect the service ﬂow, but the non-linearity in the price–z relationship may reﬂect the increasing or decreasing utility to the scale of the provision.

Possession of the characteristic in up- market models of the item may be priced at a higher rate than in a lower-priced one, i.e.

b 1 in equation (7.21).


7.88 Consider Figure 7.1 with the z characteristic being the option on the horizontal axis.

The similarity between the quantity adjustment and the option cost approaches is apparent since both relate price to some dimension of quality: the size or the option.

The option cost approach can be extended to more than one quality dimension.
Both approaches rely on the acquisition of estimates of the change in price resulting from a change in the option or size: the b slope estimates.

In the case of the quantity adjustment, this was taken from an item identical to the one being replaced, aside from the fact that it was of a diﬀerent size.

The b slope estimate in this case was perfectly identiﬁed from the two pieces of information.
It is as if the nature of the experiment controlled for changes in the other quality factors by comparing prices of what is essentially the same thing except for the quantity (size) change.

7.89 The same reasoning applies to option costs.

There may be, for example, two items, identical but for the possession of a feature.

This allows the value of the feature to be determined.

Yet sometimes the value of a feature or option has to be extracted from a much larger data set.
This may be because the quality dimension takes a relatively large range of possible numerical values without an immediately obvious consistent valuation.

Consider the simple example of only one feature varying for a product, the speed of processing of a PC.

It is not a straightforward matter to determine the value of an additional unit of speed.
To complicate matters, there may be several quality dimensions to the items and not all combinations of these may exist as items in the market 115 CONSUMER PRICE INDEX MANUAL: THEORY AND PRACTICE in any one period.

Furthermore, the combinations existing in the second period being compared may be quite diﬀerent to those in the ﬁrst.

Considering these aspects leads to a more general framework, known as the hedonic approach.
Hedonic approach 7.90 The hedonic approach is an extension of the two preceding approaches in that, ﬁrst, the change in price arising from a unit change in quality – the slope of the line in Figure 7.1 – is now estimated from a data set comprising prices and quality characteristic values of a larger number of varieties.

Second, the quality charac- teristic set is extended to cover, in principle, all major characteristics that might inﬂuence price, rather than just the quantity or option adjustment.
The theoretical basis for hedonic regressions will be covered in Chapter 21 and is brieﬂy reviewed below, following an example based on personal computers.

7.91 It should be noted that the method requires an extension of the data set to include values of the price- determining quality characteristics for each item.

Under the matched models method each price collector needed only to collect suﬃcient data on each model to allow the model to be identiﬁed for subsequent repricing.
The extension required in the hedonic approach is that all price-determining characteristics should be collected for each model.

Checklists for the characteristics of a pro- duct have been found by Merkel (2000) to improve the quality of data collected, as well as serving the needs of hedonic adjustments (see also Chapter 6 on price col- lection and Liegey, 1994).
If an item goes missing, any diﬀerence in the characteristics of its replacement can be identiﬁed and, as will be shown, a valuation can be ascribed to such diﬀerences using the hedonic approach.

7.92 Appendix 7.1 to this chapter provides data taken from the United Kingdom Compaq and Dell web sites in July 2000 on the prices and characteristics of 64 desktop personal computers (PCs).
Figure 7.2 is a scatter diagram constructed from this information, relating the price (£ sterling) to the processing speed (MHz).

It is apparent that PCs with higher speeds command higher prices – a positive relationship.
Under the option cost framework above, a switch from a 733 MHz to a 933 MHz PC would involve a measure of the slope of the Figure 7.2 Scatter diagram showing prices and processing speeds of personal computers 3500 3000 2500 2000 1500 1000 500 0 g n i l r e t s £ , e c i r P 0 200 400 600 800 1000 1200 Speed, MHz 116 line between two unique points.

The approach requires that there are 733 MHz and 933 MHz PCs that are identical except for their processing speed.
From Figure 7.2 and Appendix 7.1 it is apparent that there are several PCs with the same speed but diﬀerent prices, resulting from the fact that other things diﬀer.

To estimate the value given to additional units of speed, an estimate of the slope of the line that best ﬁts the data is required.
In Figure 7.1 the actual slope was used; for the data in Figure 7.2 an estimate of the slope needs to be derived from an estimate of the equation of the line that best ﬁts the data, using ordinary least squares regression.

Facil- ities for regression are available on standard statistical and econometric software, as well as spreadsheets.
The estimated (linear) equation in this instance is: ^PPrice= 658:436+3:261 Speed RR2=0:820 (7.22) The coeﬃcient of speed is the estimated slope of the line: the change in price (£3,261) resulting from a 1 MHz change in speed.

This can be used to estimate quality- adjusted price changes for PCs of diﬀerent speeds.

The value of RR2 indicates that 82 per cent of price vari- ation is explained by variation in processing speed.
A t-statistic to test the null hypothesis of the coeﬃcient being zero was found to be 18.83: recourse to standard tables on t-statistics found the null hypothesis was rejected at a 1 per cent level.

The fact that the estimated coeﬃcient diﬀers from zero cannot be attributed to sampling errors at this level of signiﬁcance.

There is a probability of 1 per cent that the test has wrongly rejected the null hypothesis.
7.93 The range of prices for a given speed – for example for 933 MHz – can, however, be seen from Appendix 7.1 to be substantial.

There is a price range of about £1,000, which suggests that other quality char- acteristics may be involved.

Table 7.4 provides the results of a regression equation that relates price to a number of quality characteristics using the data in Appendix 7.1.

Such estimates can be provided by standard statistical and econometric software, as well as spreadsheets.
7.94 The second column provides the results from a linear regression model, the dependent variable being price.

The ﬁrst variable is (processor) speed, with a coeﬃcient of 2.731 – a unit MHz increase in processing speed leads to an estimated £2.731 increase (positive sign) in price.

A change from 733 MHz to 933 MHz would be valued at an estimated 2002.731=£546.20.
The coeﬃcient is statistically signiﬁcant – its diﬀerence from zero (no eﬀect) not being attributable to sampling errors at a 0.1 per cent level of signiﬁcance.

This esti- mated coeﬃcient is based on a multivariate model: it is the eﬀect of a unit change in processing speed on price, having controlled for the eﬀect of other variables in the equation.

The preceding result of 3.261 in equation (7.22) was based on only one variable, and is diﬀerent from this improved result.
7.95 The brand variables are dummy intercepts taking values of 1 for, say, a Dell computer, and zero otherwise.

While brands are not in themselves quality characteristics, they may be proxy variables for other factors such as reliability or after-sales service.
The Table 7.4 Hedonic regression results for Dell and Compaq personal computers Dependent variable Price ln price ADJUSTING FOR QUALITY CHANGE 725.996 (2.71)** 2.731 (9.98)*** 1.213 (5.61) *** 4.517 (1.96)* Constant Speed (processor, MHz) RAM (random access memory, MB) HD (hard drive capacity, MB) Brand (benchmark: Compaq Deskpro) Compaq Presario Compaq Prosignia Dell Processor (benchmark: AMD Athlon) Intel Celeron Intel Pentium III ROM-drive (benchmark: CD-ROM){ CD-RW (compact disk, re-writable) DVD (digital video drive) Dell * Speed (MHz) N RR 2 { Read only memory.
Figures in parentheses are t-statistics testing a null hypothesis of the coeﬃcient being zero.

***,** and * denote statistically signiﬁcant at 0.1 per cent, 1 per cent and 5 per cent levels, respectively, tests being one-tailed.
85.539 (1.54) 1.714 (4.038)*** 63 0.934 199.506 (1.89)* 180.512 (1.38)* 1330.784 (3.74)*** 393.325 (4.38)*** 282.783 (4.28)*** 122.478 (56.07)*** 6.213 (41.95)*** 0.001364 (9.02)*** 0.000598 (5.00) *** 0.003524 (2.76)** 0.152 (2.60)** 0.167 (2.32)* 0.691 (3.52)*** 0.121 (2.43)** 0.134 (3.66)*** 0.08916 (2.88)** 0.06092 (1.99)* 0.000820 (3.49)*** 63 0.934 inclusion of such brand dummies also goes some way towards reﬂecting segmented markets as communities of buyers, as discussed in Chapter 21.
Similar dummy variables were used for the other makes or brands (Compaq Presario and Compaq Presignia), except for one brand (Compaq Deskpro) which, in this case, was taken to form the benchmark against which other models are compared.

The coeﬃcient of the Dell brand is an estimate of the diﬀerence between a Dell brand’s worth and that of a Compaq Deskpro, other variables being constant, i.e.

£1,330.78 cheaper.

Similarly, an Intel Pen- tium III commands an estimated £282.78 premium on an AMD Athlon.
7.96 The estimate for processor speed was based on data for Dell and Compaq PCs.

If the adjustment for quality is between two Dell PCs, it might be argued that data on Compaq PCs should be ignored.

Separate regressions could be estimated for each make, but this would severely restrict the sample size.

Alternatively, an interaction term or slope dummy can be used for vari- ables which are believed to have a distinctive brand- interaction eﬀect.
An example of such a dummy would be, say, Dell * Speed, which takes the value of ‘‘speed’’ when the PC is a Dell and zero otherwise.

The coeﬃcient of this variable (see Table 7.4) is 1.714; it is an estimate of the additional (positive sign) price arising for a Dell PC over and above that already arising from the stan- dard valuation of a 1 MHz increase in speed.

For Dell PCs it is £2.731+£1.714=£4.445.


Thus if the replace- ment Dell PC is 200 MHz faster than the unavailable PC, the price adjustment to the unavailable PC is to add 200 £4:465=£893.

Interactive terms for other vari- ables can similarly be deﬁned and used.

The estimation of regression equations is easily undertaken using econometric or statistical software, or data analysis facilities in spreadsheets.
An explanation of the techni- ques is given in many texts, including Kennedy (1998) and Maddala (1988).

In Chapter 21, econometric con- cerns particular to the estimation of hedonic regressions are discussed.

is 1 minus the ratio of i ^ppt i=1( pt 7.97 The value RR2 is the proportion of variation in price explained by the estimated equation.

More for- mally, the variance of it the residuals, PN i)2=N, of the equation to the i=1( pt variance of prices, PN i)2=N.
The bar on the i ppt term R2 denotes that an appropriate adjustment for degrees of freedom is made to this expression, which is necessary when comparing equations with diﬀerent numbers of explanatory variables.

At 0.934 (see Table 7.4), the value RR2 is very high.

A high value of RR2 can, however, be misleading for the purpose of quality adjustment.

First, such values indicate that the expla- natory variables account for much of the price variation.
This may be over a relatively large number of varieties of goods in the period concerned.

This, of course, is not the same as implying a high degree of prediction for an adjustment to a replacement item of a single brand in a subsequent time period.

Predicted values depend for their accuracy not just on the ﬁt of the equation, but also on how far the characteristics of the item whose price is to be predicted are from the means of the sam- ple.
The more unusual the item, the higher the predic- tion probability interval.

Second, the value RR2 indicates the proportion of variation in prices explained by the estimated equation.

It may be that 0.90 is explained while 0.10 is not explained.

If the dispersion in prices is very large, this still leaves a large absolute margin of prices unexplained.

Nonetheless, a high RR2 is a necessary condition for the use of hedonic adjustments.
7.98 Hedonic regressions should generally be con- ducted using a semi-logarithmic formulation (Chapter 21).

The dependent variable is the (natural) logarithm of the price, but the variables on the right-hand side of the equation are kept in their normal units, hence the semi-logarithmic formulation.

A double-logarithmic for- mulation would also take logarithms of the right-hand side z variables.
However, if any of these z variables are dummy variables which take the value of zero in some instances, the double-logarithmic formulation would 117 CONSUMER PRICE INDEX MANUAL: THEORY AND PRACTICE break down because logarithms of zero cannot be taken.

The focus is thus on the semi-logarithmic form.

This concern with linear and semi-logarithmic formulations is equivalent to the consideration of additive and mul- tiplicative formulations discussed in paragraphs 7.39 to 7.40 above.
A linear model would, for example, ascribe an extra £282.78 to a PC with an Intel Pentium III as opposed to an AMD Athlon, irrespective of the price of the PC.

This is common in pricing strategies using the World Wide Web.

More often than not, however, the same options are valued at a higher price for up-market goods and services.

In this case, equation (7.22) for a multivariate model becomes: Price=b0bz1 1 bz2 or ln Price= ln b0+z1 ln b1+z2 ln b2 3 .

.

.

bzn n e 2 bz3 +z3 ln b3+ .

.

.

.

.

.


zn ln bn+ ln e: (7.23) Note that this is a semi-logarithmic form; logarithms are taken of only the left-hand-side variable, i.e., price.

Each of the z characteristics enters the regression without having logarithms taken.

This has the advantage of allowing dummy variables for the possession or other- wise of a feature to be included on the right-hand side.

Such dummy variables take the value of one if the item possesses the feature and zero otherwise.
Matters relat- ing to the choice of functional form are discussed in more detail in Chapter 21.

7.99 The taking of logarithms of the ﬁrst equation (7.23) allows it to be transformed in the second equation to a linear form.

This allows the use of a conventional ordinary least squares (OLS) estimator to yield estimates of the logarithms of the coeﬃcients.
These are given in the third column of Table 7.4 and have a useful direct interpretation: if these coeﬃcients are multiplied by 100, they are the percentage change in price arising from a 1 unit change in the explanatory variable.

For (processor) speed there is an estimated 0.1364 per cent change in price for each additional MHz the replacement item has over and above the unavailable item.
When dummy variables are used, the coeﬃcients, when multiplied by 100, are estimates of the percentage change in price, given by (eb 1)100.

For example, for a rewritable CD-RW compared to a read-only CD-ROM the change in price is 8.916 per cent.

There is some bias in these coeﬃcients; and in the (semi-) logarithmic equation, half the variance of each coeﬃcient should be added to the coeﬃcient before using it (Teekens and Koerts, 1972).
For a read- only CD-ROM, the t-statistic is 2.88; this is equal to the coeﬃcient divided by its standard error, the standard error being 0.08916/2.88=0.03096 and the variance: 0.030962=0.000958.

The adjustment is to add 0.000958/2 to 0.08916, giving 0.089639 or 8.9639 per cent.

7.100 The approach is particularly useful when the market does not reveal the price of the quality char- acteristics required for the adjustment.
Markets reveal prices of items, not quality characteristics, so it is useful to consider items as tied bundles of characteristics.

A suf- ﬁciently large data set of items with their characteristics and suﬃcient variability in the mix of characteristics between the items allows the hedonic regression to pro- vide estimates of the implicit prices of the characteristics.

118 The theory behind such estimates is discussed in Chapter 21.

A number of ways of implementing the method are outlined below.
7.101 Some mention should ﬁrst be made of the interpretation of the coeﬃcients from hedonic regres- sions.

The matter is discussed in detail in Chapter 21; only the conclusions are summarized here.

There used to be an erroneous perception that the coeﬃcients from hedonic methods represented estimates of user value as opposed to resource cost.

The former is the relevant concept in constructing a consumer price index, while for PPI construction it is the latter.
Rosen (1974) found that hedonic coeﬃcients may reﬂect both user value and resource cost – both supply and demand inﬂuences.

There is what is referred to in econometrics as an iden- tiﬁcation problem; in other words, the observed data do not permit the estimation of the underlying demand and supply parameters.

Suppose that the production tech- nology of sellers is the same, but that buyers diﬀer.
Then the hedonic function describes the prices of character- istics that the ﬁrm will supply with the given ruling technology to the current mixture of tastes.

There are diﬀerent tastes on the consumer side, so what appears in the market is the result of ﬁrms trying to satisfy con- sumer preferences for a constant technology and proﬁt level.

The structure of supply is revealed by the hedonic price function.

Now suppose that sellers diﬀer, but that buyers’ tastes are the same.
Here the hedonic function p(z) identiﬁes the structure of demand.

Of these two possible assumptions, uniformity of tastes is unlikely, while uniformity of technologies is more likely, especially when access to technology is unrestricted in the long run.
Griliches (1988, p. 120) has argued in the context of a consumer price index: My own view is that what the hedonic approach tries to do is to estimate aspects of the budget constraint facing consumers, allowing thereby the estimation of ‘‘missing’’ prices when quality changes.

It is not in the business of estimating utility functions per se, though it can also be useful for these purposes .

.

.
.

.

what is being estimated is the actual locus of intersection of the demand curves of diﬀerent consumers with varying tastes and the supply curves of diﬀerent producers with possible varying tech- nologies of production.

One is unlikely, therefore, to be able to recover the underlying utility and cost functions from such data alone, except in very special circum- stances.

It is thus necessary to take a pragmatic stance.


In many cases the implicit quality adjustment to prices outlined in paragraphs 7.44 to 7.71 may be inappropriate because the implicit assumptions are unlikely to be valid.

In such instances, the practical needs of economic statistics require explicit quality adjustments.

Not to do anything on the grounds that the measures are not conceptually appropriate would be to ignore quality change and provide wrong results.
Hedonic techniques provide an important tool, making eﬀective use of data on the price– quality relationship derived from other items in the market to adjust for changes in one or more character- istics.

7.102 The proper use of hedonic regression requires an examination of the coeﬃcients of the estimated equations to see if they make sense.
It might be argued that the very multitude of distributions of tastes and technologies, along with the interplay of supply and de- mand, that determine the estimated coeﬃcients (Chapter 21) make it unlikely that ‘‘reasonable’’ estimates will arise from such regressions.

A ﬁrm may, for example, cut a proﬁt margin relating to a characteristic for reasons related to long-run strategic plans; this may yield a coeﬃcient on a desirable characteristic that may even be negative (Pakes, 2001).
This does not negate the useful- ness of examining hedonic coeﬃcients as part of a strat- egy for evaluating estimated hedonic equations.

First, there has been extensive empirical work in this ﬁeld and the results for individual coeﬃcients are, for the most part, quite reasonable.

Even over time individual coeﬃ- cients can show quite sensible patterns of decline (van Mulligen, 2003).

Unreasonable coeﬃcients on estimated equations are the exception and should be treated with some caution.
Second, one can have more faith in an estimated equation whose coeﬃcients make sense and which predicts well, than one which may also predict well but whose coeﬃcients do not make sense.

Third, if a coeﬃcient for a characteristic does not make sense, it may be due to multicollinearity, a data problem, and should be examined to see if this is the case (see Appendix 21.1 to Chapter 21).
7.103 The implementation of hedonic methods to estimate quality adjustments for matched items which are no longer available is considered below.

Consider items l, m and n where item l is available in periods t and t+2, the ‘‘old’’ item m is only available in period t and the replacement item n only in period t+2.

The items are deﬁned by their z quality characteristics, item m for example being zt m and the price of item m in period t is pt m, as depicted below.
There is no problem with comparing the prices pt of matched items with character- l l with zt+2 istics zt , for they have the same l quality characteristics.

But there is a problem with item m. A hedonic imputation approach would predict the price of item m’s characteristics in period t+2 at the character- istic prices taken from a hedonic regression estimated in period t+2, i.e.
^ppt+2 m : l and pt+2 l Item/period t l m n pt l pt m ^ppt n t+2 pt+2 l ^ppt+2 m pt+2 n m =pt n =^ppt In this case, item m’s characteristics are held constant in the comparison ^ppt+2 m. A similar exercise can be con- ducted for the replacement item n using pt+2 n. In this comparison, item n’s characteristics are held constant and compared at period t+2 and period t prices.

These imputation approaches are outlined below.

Yet there is a second approach, an adjustment one.


Here the char- acteristics of the replacement item n are identiﬁed and compared with those of the old item m, (zt+2 m), and estimated coeﬃcients from hedonic equations used to estimate the value of the changes.

These two approaches, hedonic imputations and hedonic adjustments, are con- n zt ADJUSTING FOR QUALITY CHANGE sidered below in further detail.
This ‘‘patching’’ of missing prices is quite diﬀerent from the use of hedonic price indices discussed in paragraphs 7.132 to 7.149 and Chapter 21.

These use hedonic regressions to provide hedonic price indices of overall quality-adjusted prices using a sample of all of the data in each period with no patching.
The ‘‘patching’’ of missing prices is a partial application of the hedonic approach, used in imputations for missing items or on non-comparable replacements for missing items when the matched models approach is being used and an item’s price is missing.

7.104 Hedonic imputation: Predicted vs. actual.


In this approach a hedonic regression of the natural loga- rithm of the price of model i in period t on its char- acteristics set zt ki is estimated for each month, using the equation: ln pt i=bt 0+ P bt k zt ki+et i (7.24) K k=1 Say the price of an item m available in January (period t) is unavailable in March (period t+2).
The price of item m can be predicted for March by inserting the char- acteristics of the old unavailable item m into the esti- mated regression equation for March, and similarly for successive months.

The predicted price for this old item in March and price comparison with January (period t) are given, respectively, by: 0 +P m = exp [bt+2 ^ppt+2 and ^ppt+2 m =pt m kbt+2 zt k, m] k (7.25a) That is, the old model’s price is predicted for period t+2 and patched in.
In the example in Table 7.1(a), ^pp23, ^pp24, etc.

and ^pp63, ^pp64, etc.

would be estimated and compared with p21 and p61 respectively.

The blanks for items 2 and 6 in Table 7.1(a) would be eﬀectively ﬁlled in by the estimated price from the regression equation.

7.105 An alternative procedure is to select for each unavailable m item a replacement item n. In this case the price of n in period t+2 is known, and a predicted price for n in period t is required.
The predicted price for the new item and the required price comparison are: n= exp [bt ^ppt 0+P kbt k zt+2 k, m ] and pt+2 n =^ppt n (7.25b) That is, the new model’s price is adjusted.

In this case the characteristics of item n are inserted into the right- hand side of an estimated regression for period t. The price comparisons of equation (7.25a) may be weighted by wt m, as would those of its replaced price comparison in equation (7.25b).
7.106 Another option is to take the geometric mean of the formulations in equations (7.25a) and (7.25b) on grounds analogous to those discussed in Chapter 15 and by Diewert (1997) with regard to similar index numbers.

7.107 Hedonic imputation: Predicted vs. predicted.

This approach uses predicted values for, say, item n in both periods, e.g., ^ppt+2 n. Consider a misspeciﬁcation problem in the hedonic equation.
For example, there may be an interaction eﬀect between a brand dummy and a characteristic – say for Dell and speed in the example in Table 7.4.

Possession of both characteristics n =^ppt 119 CONSUMER PRICE INDEX MANUAL: THEORY AND PRACTICE n =^ppt may be worth more in terms of price (in a semi-loga- rithmic form) than their separate individual components (for evidence of interactive eﬀects see Curry et al., 2001).
The use of pt+2 n would be misleading since the actual price in the numerator would incorporate the 5 per cent premium, while the one predicted from a straight- forward semi-logarithmic form would not.

It is stressed that, in adopting this approach, a recorded actual price is being replaced by an imputation.

This is not desirable, but neither is the form of bias discussed above.
Diewert (2002e) considers a similar problem and suggests an adjustment to bring the actual price back in line with the hedonic one.

The comparisons using predicted values in both periods are given as: does not have to be updated in each period.

However, (7.25b) and (7.27b) eﬀectively compare a constant ﬁxed basket of current period characteristics while (7.25a) and (7.27a) compare a ﬁxed basket of price reference period characteristics.
There is no reason to prefer one to the other and if the diﬀerence or spread between the two indices is large, this is reason for caution over the use of one against a geometric mean of the two.

Regular updating of hedonic regressions would be likely to minimize spread.

7.109 Hedonic: Indirect adjustment.


An indirect adjustment may be made for the current period, which only requires the hedonic regression to be estimated in the base period t, using: ^ppt+2 n =^ppt ^ppt+2 m =^ppt [( ^ppt+2 n for the new item m for the disappearing or old item; or n =^ppt n )( ^ppt+2 m =^ppt m )]1=2 (7.26) as a geometric mean of the two.

7.108 Hedonic adjustments.


In this approach a replacement item is used and any diﬀerences between the k characteristics of the replacement n in, for example, period t+2 and m in period t are ascertained.
A pre- dicted price for m adjusted to be compatible with n is estimated for period t, i.e., ^ppt+2 m and is compared with the actual price, pt m, where n k kbt mk)] mk)] (zt+2 exp [ P (7.27a) (7.27b) , where n pt ^ppt nk zt m pt+2 ^ppt+2 n nk zt kbt+2 or alternatively, a predicted price for n adjusted to be compatible with m is estimated for period t, i.e.
^ppt n is compared with the actual price, pt+2 m exp [P k(zt+2 The adjustments here are undertaken using predicted values.
However, unlike the formulations in equation (7.27b), for example, ^ppt n may be estimated by applying the subset of the k characteristics that distinguished m from n to their respective implicit prices in period t estimated from the hedonic regression, and adjusting the price of pt m. For example, if the nearest replacement for item 2 is item 3, then the characteristics that diﬀer- entiated item 3 from item 2 are identiﬁed and the price in the base period, p31, is estimated by adjusting p21 using the appropriate coeﬃcients from the hedonic regression in that month.
For example, for washing machines, if item 2 had a spin speed of 800 rpm and item 3 a spin speed of 1,100 rpm, other things being equal, the shadow price of the 300 rpm diﬀerential would be estimated from the hedonic regression and p21 would be adjusted for comparison with p33.

Note that if the z variables in the characteristic set are perfectly independent of each other, the results from this approach will be similar to those from equation (7.25).
This is because inter- dependence between the variables on the right-hand side of the hedonic equation – multicollinearity – leads to imprecise estimates of the coeﬃcients (see Chapter 21).

Hedonic imputations and adjustments of the form (7.25b) and (7.27b) have an advantage over their coun- terparts (7.25a) and (7.27a) since the regression equation 120 pt+2 n pt m ^ppt n ^ppt m (7.28) The ﬁrst term is the change in price between the old and replacement items in periods t and t+2 respectively.
But the quality of the item has also changed, so this price change needs to be divided by a measure of the change in quality.

The second term uses the hedonic regression in period t in both the numerator and denominator.

The coeﬃcients – the shadow prices of each characteristic – are held constant.
The predicted prices nevertheless diﬀer because diﬀerent quantities of the characteristics are being inserted into the numerator and denominator: the characteristics of the replacement item n in the former and the old item m in the latter.

The measure is the change in price after removing (by division) the change in quantity of characteristics for each item at a constant period t price.
Of course, conceptually, the constant valuation by a period t+2 regression would be equally valid and a geometric mean of the two ideal.

However, if hedonic regressions cannot be run in real time this is a compromise.

As the spread between the current and base period results increases, its validity decreases.

As such, the regression estimates should be updated regularly using old and current period estimates and results com- pared retrospectively as a check on the validity of the results.
Limitations of the hedonic approach 7.110 The limitations of the hedonic approach should be borne in mind.

Some points are summarized below (see also Chapter 21).

First, the approach requires statistical expertise for the estimation of the equations.

The availability of user-friendly software with regression facilities makes this less problematic.

Statistical and econometric software carries a range of diagnostic tests to help judge if the ﬁnal formulation of the model is satisfactory.
These include RR2 as a measure of the overall explanatory power of the equation, and F-test and t-test statistics to enable tests to be conducted as to whether the diﬀerences between the coeﬃcients of the explanatory variables are jointly and individually diﬀerent from zero at speciﬁed levels of statistical signiﬁcance.

Most of these statistics make use of the errors from the estimated equation.
The regression equation can be used to predict prices for each item by inserting the values of the characteristics of the items into the explanatory vari- ables.

The diﬀerences between the actual prices and these predicted results are the residual errors.
Bias or impre- cise, and thus misleading, results may arise from a range of factors including heteroscedasticity (non-constant variances in the residuals suggesting non-linearities or omission of relevant explanatory variables), a non- normal distribution for the errors, and multicollinearity, where two or more explanatory variables are related.

The latter in particular has been described as the ‘‘bane of hedonic regressions’’ (Triplett, 1990).
Such econometric issues have been discussed in the context of hedonic regressions (Berndt, 1991; Berndt et al., 1995; Triplett, 1990; Gordon, 1990; Silver, 1999; and in Chapter 21) and more generally by Kennedy (1998) and Maddala (1988).

For the reasons discussed above, when multicollinearity is suspected, the use of predicted values rather than individual coeﬃcients is advised.

7.111 Second, the estimated coeﬃcients should be updated regularly.
If the adjustment is to the old model, then the price comparison is between the price of the old model in some reference period adjusted for the quality diﬀerence between the old and new models, using coef- ﬁcients from an estimated hedonic equation in the price reference period as estimates of the value of such dif- ferences, as in (7.27b).

There is, at ﬁrst sight, no need to update the estimated coeﬃcients each month.
Yet the valuation of a characteristic in the price reference period may be quite out of line with its valuation in the new period.

For example, a feature may be worth an addi- tional 5 per cent in the reference period instead of 10 per cent in the current period, because it might have been introduced at a discount at that point in its life cycle to encourage usage.
Continuing to use the coeﬃcients from some far-oﬀ period to make adjustments to prices in the current period is akin to using out-of-date base period weights.

The comparison may be well deﬁned, but have little meaning.
If price adjustments for quality diﬀer- ences are being made to the old item in the price refer- ence period using hedonic estimates from that period, then there is a need to update the estimates if they are considered to be out of date, say because of changing tastes or technology, and splice the new estimated comparisons onto the old.

The regular updating of hedonic estimates when using imputations or adjust- ments is thus recommended, especially when there is evidence of parameter instability over time.
Ideally a geometric mean of either (7.25a) and (7.25b) or of (7.27a) and (7.27b) should be used, but this requires an updating of hedonic regressions in real time.

7.112 Third, the sample of prices and characteristics used for the hedonic adjustments should be suitable for the purpose.
If they are taken from a particular outlet or outlet type, trade source or web page and then used to adjust non-comparable prices for items sold in quite diﬀerent outlets, then there must at least be an intu- ition that the marginal utilities for characteristics are similar between the outlets.

A similar principle applies for the brands of items used in the sample for the hedonic regression.

It should be borne in mind that high RR2 statistics do not alone ensure reliable results.
Such high values arise from regressions in periods prior to ADJUSTING FOR QUALITY CHANGE their application and indicate the proportion of vari- ation in prices across many items and brands.

They are not in themselves a measure of the prediction error for a particular item, sold in a speciﬁc outlet, of a given brand in a subsequent period, though they can be an important constituent of this.

7.113 Fourth, there is the issue of functional form and the choice of variables to include in the model.
Simple functional forms generally work well.

These include linear, semi-logarithmic (logarithm of the left- hand side) and double-logarithmic (logarithms of both sides) forms.

Such issues are discussed in Chapter 21.

The speciﬁcation of a model should include all price-deter- mining characteristics.

Some authors advise quite simple forms with only the minimum number of variables, as long as the predictive capacity is high (Koskima¨ ki and Vartia, 2001).
Shepler (2000) included 33 variables in her hedonic regressions of refrigerators – a fairly homo- geneous product.

These included nine dummy variables for brand and four for colour, ﬁve types of outlets, three regions as control variables, and 11 characteristics includ- ing capacity, types of ice-maker, energy-saving control, extra drawers, sound insulation, humidiﬁer and ﬁl- tration device.
Typically, a study would start with a large number of explanatory variables and a general econo- metric model of the relationship, while the ﬁnal model would be more speciﬁc, having dropped a number of variables.

The dropping of variables would depend on the result of experimenting with diﬀerent formulations, and seeing their eﬀects on diagnostic test statistics, including the overall ﬁt of the model and the accordance of signs and magnitudes of coeﬃcients with prior expectations.
Reese (2000), for example, started with a hedonic regression for United States college textbooks which included about 50 explanatory variables, subse- quently reduced to 14 such variables with little loss of explanatory power.
7.114 Finally, Bascher and Lacroix (1999) list sev- eral requirements for successful design and use of hedonic quality adjustment in the consumer price index, noting that these require heavy investments over a long period involving: intellectual to develop and re-estimate the model, and to employ it when products are replaced; competencies and suﬃcient time access to detailed, reliable information on product characteristics; a suitable organization of the infrastructure for col- lecting, checking and processing information.
7.115 Hedonic methods may also improve quality adjustment in the consumer price index by indicating which product attributes do not appear to have material impacts on price.

That is, if a replacement item diﬀers from the old item only in characteristics that have been rejected as price-determining variables in a hedonic study, this would support a decision to treat the items as com- parable or equivalent to and include the entire price dif- ference, if any, as pure price change.
Care has to be exercised in such analysis because a feature of multi- collinearity in regression estimates is that the imprecision of the parameter estimates may give rise to statistical tests 121 CONSUMER PRICE INDEX MANUAL: THEORY AND PRACTICE that do not reject null hypotheses that are false, i.e., they do not ﬁnd signiﬁcant parameter estimates that are sig- niﬁcant.
The results from such regressions can none- theless provide valuable information on the extent to which diﬀerent characteristics inﬂuence price variation, and this in turn can help in the selection of replacement items.
Enhanced conﬁdence in item substitution and the quality adjustment of prices that arises from using the hedonic approach, and the parallel reduction in reliance on ‘‘linking’’, has been cited as a signiﬁcant beneﬁt in terms of the reliability of the measurement of price changes for apparel in the United States consumer price index (Reinsdorf, Liegey and Stewart, 1996).
The results from hedonic regressions have a role to play in identifying price-determining characteristics and may be useful in the design of quality checklists in price collection (Chapter 6).

Choice between quality adjustment methods 7.116 Choice of method for quality adjustments to prices is not straightforward.

The analyst must consider the technology and market for each commodity and devise appropriate methods.
This is not to say the methods selected for one product area will be indepen- dent of those selected for other areas.

Expertise built up using one method may encourage its use elsewhere, and intensive use of resources for one commodity may lead to less resource-intensive methods for others.
The methods adopted for individual product areas may vary between countries as access to data, relationships with the outlet managers, resources, expertise and features of the pro- duction, and market for the product vary.

Guidelines on choice of method arise directly from the features of the methods outlined above.

A good understanding of the methods, and their implicit and explicit assumptions, is essential to the choice of an appropriate method.
7.117 Figure 7.3 provides a guide to the decision- making process.

Assume that the matched models method is being used.

If the item is matched for re- pricing, there being no change in the speciﬁcation, no quality adjustment is required.

This is the simplest of procedures.

However, a caveat applies.

If the item belongs to a high-technology industry where model replacement is rapid, the matched sample may become unrepresentative of the universe of transactions.
Alter- natively, matching may be under a chained framework, where prices of items in a period are matched to those in the preceding period to form a link.

A series of successive links of matched comparisons combined by successive multiplication makes up the chained matched index.

Or hedonic indices may be used which require no matching.

The use of such methods is discussed in paragraphs 7.132 to 7.149.

At the very least, attention should be directed to more regular item re-sampling.
Continued long-run matching would deplete the sample and an alternative framework to long-run matching would be required.

7.118 Consider a change in the quality of an item and assume that a replacement item is available.

The selec- tion of a comparable item to the same speciﬁcation and the use of its price as a comparable replacement require 122 that none of the price diﬀerence is attributable to quality.

It also requires conﬁdence that all price-determining factors are included in the speciﬁcation.
The replacement item should also be representative and account for a reasonable proportion of sales.

Caution is required when replacing near obsolete items with unusual pricing at the end of their life cycles with similar ones that account for relatively low sales, or with ones that have quite sub- stantial sales but are at diﬀerent points in their cycle.

Strategies for ameliorating such eﬀects are discussed below and in Chapter 8, including early substitutions before pricing strategies become dissimilar.
7.119 Figure 7.3 illustrates the case where quality diﬀerences can be quantiﬁed.

Explicit estimates are gen- erally considered to be more reliable, although they are also more resource intensive, at least initially.

Once an appropriate methodology has been developed, they can often be easily replicated.

General guidelines are more diﬃcult here as the choice depends on the host of factors discussed above, which are likely to make the estimates more reliable in each situation.
Central to all of this is the quality of the data upon which the estimates are based.

If reliable data are unavailable, subjective judgements may be used.

Product diﬀerences are often quite technical and very diﬃcult to specify and quantify.

The reliability of the method depends on the expertise of the experts and the variance in opinions.

Estimates based on objective data are thus preferred.
Good production cost estimates in industries with stable technologies and identiﬁable con- stant retail mark-ups and where diﬀerences between the old and replacement items are well speciﬁed and exhaus- tive are, by deﬁnition, reliable.

Estimates of the retail mark-up are, however, prone to error and the option cost approach is generally preferable.

This requires that the old and new items diﬀer by easily identiﬁable character- istics which are or have been separately priced as options.
7.120 The use of hedonic regressions for partial patching is most appropriate where data on price and characteristics are available for a range of models and where the characteristics are found to predict and explain price variability well in terms of a priori reasoning and econometric terms.

Their use is appropriate where the cost of an option or change in characteristics cannot be separately identiﬁed and has to be gleaned from the prices of items sold with diﬀerent speciﬁcations in the market.
The estimated regression coeﬃcients are the estimate of the contribution to price of a unit change in a characteristic, having controlled for the eﬀects of vari- ations in the quantities of other characteristics.

The estimates are particularly suited to valuing changes in the quality of an item when only a given set of characteristics changes and the valuation is required for changes in these characteristics only.
The results from hedonic regressions may be used to target the salient characteristics for item selection.

The synergy between the selection of prices according to characteristics deﬁned as price determin- ing by the hedonic regression, and their subsequent use for quality adjustment, should reap rewards.

The method should be applied where there are high ratios of non-comparable replacements and where the diﬀerences between the old and new items can be well deﬁned by a large number of characteristics.
Figure 7.3 Flowchart for making decisions on quality change Yes Has the specification changed?

No ADJUSTING FOR QUALITY CHANGE Continue to use matched models Has the quality changed?

No Yes Can the quality difference be explicitly quantified?

Yes Is a replacement available?


Expert judgement No Yes − use an explicit adjustment Quantity adjustment Replacement available: assume all price difference is attributable to quality No replacement Replacement available: assume none of price difference is attributable to quality Are the old and new varieties available simultaneously?
Yes No Production cost Option costs Hedonic techniques Comparable replacement Overall/targeted/class mean imputation Carry-forward Overlap pricing Linked to show no change Source: Chart developed from a version by Fenella Maitland-Smith and Rachel Bevan, OECD; see also a version in Triplett (2002).

7.121 If explicit estimates of quality are unavailable, and no replacement items are deemed appropriate, then imputations may be used.

The use of imputations has much to commend it resource-wise.
It is relatively easy to employ – though some veriﬁcation of the validity of the implicit assumptions might be appropriate.

It requires no judgement (unless targeted) and is therefore objective.

Targeted mean imputation is preferred to overall mean imputation as long as the sample size upon which the target is based is adequate.
Class mean imputation is preferred when models at the start of their life cycles are replacing those around the end of their life cycle, 123 CONSUMER PRICE INDEX MANUAL: THEORY AND PRACTICE although the approach requires faith in the adequacy of the explicit and comparable replacements being made.
7.122 Bias from using imputation is directly related to the proportion of missing items and the diﬀerence between quality-adjusted prices of available matched items and the quality-adjusted prices of unavailable ones (see Table 7.2 on page 110).

The nature and extent of the bias depends on whether short-run or long-run imputa- tions are being used (the former being preferred) and on market conditions (see paragraphs 7.159 to 7.173).
Imputation, in practical terms, produces the same result as deletion of the item.

The inclusion of imputed prices may give the illusion of larger sample sizes.

Imputation is less likely to introduce bias where the proportion of missing prices is low.

Table 7.2 can be used to estimate likely error margins arising from its use and a judgement can be made as to whether they are acceptable.
The use of imputation across many products need not necessarily compound the errors since, as noted in the above dis- cussion of this method, the direction of bias need not be systematic.

It is cost-eﬀective for product areas with a large number of missing items because of its ease of use.

But the underlying assumptions required by imputation must be very carefully considered if it is widely used.
Imputation should by no means be the overall catch- all strategy, and statistical agencies are advised against its use as a default device without due consideration of the nature of the markets, the possibility of target- ing the imputation and the viability of estimates from the sample sizes involved if such targeting is employed.
If the old and replacement items are available simultaneously, and if the quality diﬀerence cannot be quantiﬁed, an implicit approach can be used whereby the price diﬀerence between the old and replacement items in a period in which they both exist is assumed to be attributable to quality.

This overlap method, in repla- cing the old item by a new one, takes the ratio of prices in a period to be a measure of their quality diﬀerence.

It is implicitly used when new samples of items are taken.
The assumption of relative prices equating to quality diﬀer- ences at the time of the splice is unlikely to hold if the old and replacement items are at diﬀerent stages in their life cycles and diﬀerent pricing strategies are used at these stages.

For example, there may be deep discounting of the old item to clear inventories, and price skimming of market segments that will purchase new models at rela- tively high prices.
As with comparable replacements, early substitutions are advised so that the overlap is at a time when items are at similar stages in their life cycles.

7.123 7.124 For the reasons discussed, the use of the linked to show no change method and the carry-forward method is not generally advised for making quality adjustment imputations, unless the implicit assumptions are deemed to be valid.
High-technology and other sectors with a rapid turnover of models 7.125 The measurement of price changes of items unaﬀected by quality changes is primarily achieved by matching models, the above techniques being applicable 124 when the matching breaks down.

But what of industries where the matching breaks down on a regular basis because of the high turnover in new models of diﬀerent qualities to the old ones?
The matching of prices of identical models over time, by its nature, is likely to lead to a seriously depleted sample.

There is both a dynamic universe of all items consumed and a static universe of the items selected for repricing (Dale´ n, 1998a).
If, for example, the sample is initiated in December, by the subsequent May the static universe will be matching prices of those items available in the static universe in both December and May, but will omit the unmatched new items introduced in January, February, March, April and May, and the unmatched old ones available in December but unavailable in May.

Two empirical questions show whether there will be any signiﬁcant bias.

First, is sample depletion substantial?


Substantial depletion of the sample is a necessary condition for such bias.

Second, are the unmatched new and unmatched old items likely to have quality-adjusted prices that sub- stantially diﬀer from those of the matched items in the current and the base periods?

is increasingly unrepresentative of 7.126 The matching of prices of identical models over time may lead to the monitoring of a sample of models that the population of transactions.
Some of the old models that existed when the sample was drawn are not available in the current period; and new models that enter the sample are not available in the base period.

It may be that the models that are going out have relatively low prices, while the entrants have relatively high ones.

By ignoring these prices, a bias is being introduced.

Using old low- priced items and ignoring new high-priced ones has the eﬀect of biasing the index downwards.
In some indus- tries, the new item may be introduced at a relatively low price and the old one may become obsolete at a relatively high price, serving a minority segment of the market (Berndt et al., 2003).

In this case, the bias would take the opposite direction.

The nature of the bias will depend on the pricing strategies of ﬁrms for new and old items.

7.127 This sampling bias exists for most products.


Our concern here, however, is with product markets where the statistical agencies are ﬁnding the frequency of new item introductions and old item obsolescence suﬃ- ciently high that they may have little conﬁdence in their results.

First, some examples of such product markets will be given and then two procedures will be considered: the use of hedonic price indices (as opposed to the par- tial, hedonic patching discussed above) and chaining.
Some examples 7.128 Koskima¨ ki and Vartia (2001) attempted to match prices of models of personal computers (PCs) over three two-month periods (spring, summer and autumn) using a sample of prices collected as part of standard price collection for the Finnish consumer price index.

Of the 83 spring prices, only 55 matched comparisons could be made with the summer prices, and then only 16 con- tinued through to the autumn.
The sample of matched pairs became increasingly rapidly biased: of the 79 models in the autumn, the 16 matched ones had a mean processor speed of 518 MHz compared with 628 MHz for the remaining 63 unmatched ones; the hard disk sizes were, respectively, 10.2 and 15.0 Gigabytes, and the percentages of high-end processors (Pentium III and AMD Atl.)

were 25 per cent and 49.2 per cent, respec- tively.
Hardly any change in matched prices was found over this six-month period, while a hedonic regression analysis using all of the data found quality-adjusted price falls of around 10 per cent.

Instructions to price collec- tors to hold onto models until forced replacements are required may thus lead to a sample that is increasingly unrepresentative of the population and is biased towards technically inferior variants.
In this instance, the hedonic price changes fell faster since the newer models became cheaper for the services supplied.

7.129 Kokoski et al.

(1999) used hedonic regressions in an empirical study of inter-area price comparisons of food products across urban areas in the United States using the United States consumer price index data.
They found a negative sign on the coeﬃcients of dummy variables for whether or not the sample items were from newly rotated samples (dummy variable=1) or samples prior to rotation (dummy variable=0).

This indicated that quality-adjusted prices were lower for the newly included items compared with the quality-adjusted pri- ces of the old items.

7.130 Silver and Heravi (2002) found evidence of sample degradation when matching prices of United Kingdom washing machines over a year.
By December, only 53 per cent of the January basket of model varieties was used for the December/January index, although this accounted for 81.6 per cent of January expenditure.

Models of washing machines with lower sales values dropped out faster.

However, the remaining models in December accounted for only 48.2 per cent of the value of transactions in December.

The active sample relating to the universe of transactions in December had substantially deteriorated.
The prices of unmatched and matched models were found to diﬀer, as were their vintage and quality.

Even when prices were adjusted for quality using hedonic regressions, prices of unmatched old models were found to be lower than matched ones, there also being evidence of higher prices for unmatched new models.

Quality-adjusted prices fell faster for the matched sample than for the full sample: about 10 per cent compared with about 7 per cent.
Residuals from a common hedonic sur- face and their leverage were also examined.

The residuals from unmatched new models were higher than matched ones, while residuals from unmatched old models were much lower.

Unmatched observations had nearly twice the (unweighted) leverage as matched ones – their inﬂuence in the estimation of the parameters of the regression equation was much greater, and their exclusion more serious.
7.131 The above studies demonstrate how serious sample degradation can occur and how unmatched excluded items may be quite diﬀerent from included ones.

Two procedures for dealing with such situations will be considered: the use of hedonic price indices (as opposed to the partial, hedonic patching discussed above) and chaining.

Both rely on a data set of a repre- sentative sample of items and their characteristics in each period.
Price collectors might use a checklist of char- ADJUSTING FOR QUALITY CHANGE acteristics in gathering the data (Merkel, 2000).

They will be asked to collect prices and characteristics of more than one item in each store, the items being the major or typical ones sold.

If a new item is introduced which has or is likely to have substantial sales, then it is included as a replacement or even addition, and its characteristics are marked oﬀ against a checklist of salient characteristics.
The list will be developed at the time of initiating the sample, and updated as required.

Alternatively, market research agencies, web pages and trade associations may also be able to provide lists of models and their prices.

Nevertheless, there is a need to collect transaction prices, as opposed to list prices.
Hedonic price indices 7.132 It is important to distinguish between the use of hedonic regressions to make adjustments for quality diﬀerences when a non-comparable substitute is used, as in paragraphs 7.90 to 7.115, and their use in their own right as hedonic price indices, which are measures of quality-adjusted price changes.
Hedonic price indices are suitable when the pace and scale of replacements of items are substantial because, ﬁrst, an extensive use of quality adjustments may lead to errors and, second, the sampling will be from a matched/replacement universe likely to be biased.

With new models being continually introduced and old ones disappearing, the coverage of a matched sample may deteriorate and bias may be introduced as the price changes of new/old models diﬀer from those of the matched ones.
What is required is a sample to be drawn in each month and price indices constructed; but instead of controlling for quality dif- ferences by matching, they will be controlled for, or ‘‘partialled out’’, in the hedonic regression.

Note that all the indices described below use a fresh sample of the data available in each period.

If there is a new item in a period, it is included in the data set and its quality dif- ferences controlled for by the regression.
Similarly, if old items drop out, they are still included in the data for the indices in the periods in which they exist.

Paragraphs 7.110 to 7.115 stress the need for caution in the use of hedonic regressions for quality adjustments; some of the theoretical and econometric aspects are considered in Chapter 21.

This need for caution extends to the use of the results from hedonic indices, and the discussion is not repeated here for the sake of brevity.
7.133 In Chapter 17, theoretical price indices are deﬁned and practical index number formulae are con- sidered as bounds or estimates of these indices.

Theoretical index numbers are also deﬁned in Chapter 21 to include goods made up of tied characteristics, so something can be said about how such theoretical indices relate to dif- ferent forms of hedonic indices.

A number of forms are considered in Chapter 21; they are summarized below.

7.134 Hedonic functions with dummy variables for time.
The sample covers the two time periods being compared, say t and t+2, and does not have to be matched.

The hedonic formulation regresses the price of item i, pi, on the k=2; .

.

.

K characteristics of the items zki.


A single regression is estimated on the data in the two time periods compared, the equation also including a dummy 125 CONSUMER PRICE INDEX MANUAL: THEORY AND PRACTICE variable Dt+2 being 1 in period t+2, zero otherwise: ln pi=b0+b1Dt+2+ P K k=2 bkzki+ei (7.29) The coeﬃcient b1 is an estimate of the quality-adjusted price change between period t and period t+2.
It is an estimate of the change in the logarithm of price, having controlled for the eﬀects of variation in quality via PK k=2bkzki.

Note that an adjustment is required for b1: the addition of ½ (standard error)2 of the estimate, as discussed in Goldberger (1968) and Teekens and Koerts (1972).

Two variants of equation (7.28) are considered.

The ﬁrst is the direct ﬁxed base version, that compares period t with t+2 as outlined: January–February, January–March, etc.
The second is a rolling chained version evaluated for period t with t+1; then again for t+1 with t+2, the links in the chain being combined by successive multiplication.

A January–March compar- ison, for example, would be the January–February index multiplied by the February–March one.
There is, of course, a fully constrained version: a single constrained regression for, say, January to December with dummy variables for each month, but this is impractical in real time since it requires data on future observations.

7.135 The above approach uses the dummy vari- ables on time to compare prices in period 1 with prices in each subsequent period.

In doing so, the b parameters are constrained to be constant over the period being compared.
A ﬁxed base, bilateral comparison using equation (7.29) makes use of the constrained parameter estimates over the two periods and, given an equal number of observations in each period, is a form of a symmetric average.

A chained formulation would esti- mate I1,4, for example, as: I1,4=I1,2 I2,3 I3,4.

In each binary comparison for matched data, equal weight is also given to the data in each period.
7.136 There is no explicit weighting in these for- mulations and this is a serious disadvantage.

In practice, ‘‘cut-oﬀ ’’ sampling might be employed to include only the most important items.

If sales data are available, a weighted least squares (WLS) estimator should be used, as opposed to an ordinary least squares (OLS) estimator.
It is axiomatic in normal index number construction that the same weight should not be given to each price com- parison, since some items may account for much larger sales revenues than others.

The same consideration applies to these hedonic indices.

Diewert (2002e) has argued for a preference for sales value weights over quantity weights.
Two items may have sales equal to the same quantity, but if one is priced higher than another, its price changes should be accordingly weighted higher for the result to be meaningful in an economic sense.
Addi- tionally, Diewert (2002e) has shown that value shares should form the weights, since values will increase, in say period t+2, with prices, the residuals and their variance thus being higher in period t+2 than in t. This hetero- scedasticity is an undesirable feature of a regression model, resulting in increased standard errors.
Silver (2002) has further shown that a WLS estimator does not the observations by their designated purely weight weights, the actual inﬂuence given being also the result of 126 a combination of the residuals and the leverage eﬀect.

The latter is higher as the characteristics of the observations diverge from the average characteristics of the data.

Silver suggests that observations with relatively high leverage and low weights be deleted and the regression re-run.

7.137 Period-to-period hedonic indices.


An alter- native approach for a comparison between periods t and t+2 is to estimate a hedonic regression for period t+2, and insert the values of the characteristics of each model existing in period t into the period t+2 regression to predict, for each item, its price.

This would generate predictions of the prices of items existing in period t based on their zt i characteristics, at period t+2 shadow prices, ^ppt+2 i).
These prices (or an average) can be compared with the actual prices (or the average of pri- ces) of models in period t, pt i(zt i) as, for example, a Jevons hedonic base period index: (zt i 1=Nt ^ppt+2 i (zt i) 1=Nt = i(zt pt i) Nt Q i=1 1=Nt (zt i) ^ppt+2 i 1=Nt ^ppt i Nt Q i=1 1=Nt ^ppt+2 i (zt i) PJHB= = Nt Q i=1 Nt Q i=1 Nt Q i=1 1=Nt pt i (7.30a) Nt Q i=1 7.138 Alternatively, the characteristics of models existing in period t+2 can be inserted into a regression for period t. Predicted prices of period t+2 items gen- erated at period t shadow prices, pt ), are the prices of items existing in period t+2 estimated at period t prices and these prices (or an average) can be compared with the actual prices (or the average of prices) in period t+2, pt+2 ); a Jevons hedonic current period index is: i(zt+2 i (zt+2 i i PJHC= = #1=Nt+2 ) 1=Nt+2 = ) " Nt+2 Q i=1 pt+2 i (zt+2 i Nt+2 Q i=1 " N+2t Q i=1 i(zt+2 pt i #1=Nt+2 pt+2 i 1=Nt+2 Nt+2 Q i=1 i(zt+2 pt i ) #1=Nt+2 " Nt+2 Q i=1 ^ppt+2 i 1=Nt+2 Nt+2 Q i=1 i (zt+2 pt i ) (7.30b) 7.139 For a ﬁxed base bilateral comparison using either equation (7.30a) or equation (7.30b), the hedonic equation is only estimated for one period, the current period t+2 in equation (7.30a) and the base period t in to those equation (7.30b).
For explained in Chapters 15, 16 and 17, a symmetric average of these indices would have some theoretical support.

reasons analogous 7.140 Note that a geometric mean of (7.30) uses all the data available in each period, as does the hedonic index using a time dummy variable in (7.29).

If in (7.29) there is a new item in, say, period t+2, it is included in the data set and its quality diﬀerences controlled for by the regression.
Similarly, if old items drop out, they are still included in the indices in the periods in which they exist.

This is part of the natural estimation procedure, unlike using matched data and hedonic adjustments on non- comparable replacements when items are no longer available.

7.141 With the dummy variable approach, there is no explicit weighting in its formulation in (7.29), and this is a serious disadvantage.
In practice, cut-oﬀ sam- pling might be employed to include only the most important items; or if expenditure data are available, a WLS as opposed to an OLS estimator might be used, with expenditure value shares as weights, as discussed in Appendix 21.1 to Chapter 21.

7.142 Superlative and exact hedonic indices (SEHI).

In Chapter 17, Laspeyres and Paasche bounds are deﬁned on a theoretical basis, as are superlative indices, which treat both periods’ data symmetrically.
These superlative formulae, in particular the Fisher index, are also seen in Chapter 16 to have desirable axiomatic properties.

Fur- thermore, the Fisher index is supported by economic theory as a symmetric average of the Laspeyres and Paasche bounds, being found to be the most suitable such average on axiomatic grounds.

The To¨ rnqvist index is seen to be best from the stochastic viewpoint, and also does not require strong assumptions for its derivation from the economic approach as a superlative index.
The Laspeyres and Paasche indices are found to correspond to (be exact for) underlying Leontief aggregator functions with no substitution possibilities, while superlative indi- ces are exact for ﬂexible functional forms, including the quadratic and translogarithmetic forms for the Fisher and To¨ rnqvist indices, respectively.

If data on prices, characteristics and quantities are available, analogous approaches and ﬁndings arise for hedonic indices (Fixler and Zieschang, 1992 and Feenstra, 1995).
Exact theore- tical bounds on a hedonic index have been deﬁned by Feenstra (1995).

Consider index in Chapter 17, equation (17.3), but now deﬁned only over items in terms of their characteristics zi.

The prices (and quantities) are still of items, but they are wholly deﬁned through their characteristics pi (zi).
An arithmetic aggre- gation for a linear hedonic equation ﬁnds a Laspeyres upper bound (as quantities demanded decrease with increasing relative prices) given by: the theoretical N P i=1 N P i=1 i ^ppt+2 qt i ipt qt i N =P i=1 st i ^ppt+2 i pt i C(ut, p(z)t+2) C(ut, p(z)t) (7.31a) where the right-hand-side expression is the ratio of the cost of achieving a period t level of utility (ut), where utility is a function of the vector of quantities, i.e., ut=f (qt).
The price comparison is evaluated at a ﬁxed level of period t quantities, and st i are the shares in total in period t, value of expenditure on product i=qt st i and i=PN i=1qt ipt ipt i ADJUSTING FOR QUALITY CHANGE are prices in period t+2 adjusted for the sum of the changes in each quality characteristic weighted by their coeﬃcients derived from a linear hedonic regression.
Note that the summation is over the same i in both periods, since replacements are included when an item is missing and equation (7.31b) adjusts their prices for quality diﬀerences.
7.143 A Paasche lower bound is estimated as: N P i=1 N P i=1 qt+2 i pt+2 i qt+2 i ^ppt i N = P i=1 st+2 i pt+2 i ^ppt i 1 C(ut+2, p(z)t+2) C(ut+2, p(z)t) where st+2 i =qt+2 i pt+2 i qt+2 i pt+2 i and N = P i=1 N i=1 i pt ^ppt i+P bt k(zt+2 ik zt ik) (7.32a) (7.32b) which are prices in periods t adjusted for the sum of the changes in each quality characteristic weighted by its respective coeﬃcients derived from a linear hedonic regression.
7.144 In Chapter 17 it is shown that Laspeyres PL and Paasche PP price indices form bounds on their respective ‘‘true’’ economic theoretic indexes.
Using similar reasoning to that in Chapter 17 applied to equations (7.31a) and (7.32a), it can be shown that under homothetic preferences these true economic indices col- lapse into a single theoretical index c( pt+2)/c( pt), and: PL c( pt+2)=c( pt) PP (7.33) 7.145 The approach is akin to that used for adjust- ments to non-comparable replacement items in equa- tions (7.27a) and (7.27b), above.
However, the SEHI approach ﬁrst uses all the data in each period, not just the matched sample and selected replacements.

Second, it uses the coeﬃcients from hedonic regressions on changes in the characteristics to adjust observed prices for quality changes.

Third, it incorporates a weighting system using data on the expenditure shares of each model and their characteristics, rather than treating each model as equally important.
Finally, it has a direct correspondence to formulations deﬁned from economic theory.
7.146 Semi-logarithmic hedonic regressions would supply a set of b coeﬃcients suitable for use with these base and current period geometric bounds: s t+2 i N Q i=1 pt+2 i ^ppt i C(u; p(z)t+2) C(u; p(z)t) s t i N Q i=1 ^ppt+2 i pt i (7.34a) i pt ^ppt i exp P N bt k(zt+2 ik zt ik) i=1 i pt+2 ^ppt+2 i N exp P i=1 bt+2 k (zt+2 ik zt ik) (7.34b) i pt+2 ^ppt+2 i P N i=1 bt+2 k (zt+2 ik zt ik) (7.31b) 7.147 In equation (7.34a) the two bounds on the respective theoretical indices have been shown to be brought together under an assumption of homothetic 127 CONSUMER PRICE INDEX MANUAL: THEORY AND PRACTICE preference (see Chapter 17).
The calculation of such indices is no small task.

For examples of their applica- tion, see Silver and Heravi (2001a and 2003) for com- parisons over time and Kokoski et al.

(1999) for price comparisons across areas of a country.

Kokoski et al.

(1999) used a sample from a replacement universe of otherwise matched data from the United States Bureau of Labor Statistics consumer price index, though the sample beneﬁted from rotation.
Silver and Heravi (2001a and 2003) used scanner data for the universe of trans- actions via a two-stage procedure in which cells were deﬁned according to major price-determining features such as all combinations of brand, outlet type and (for television sets) screen size – much like strata.

There may be a gain in the eﬃciency of the ﬁnal estimate since the adjustment is for within-strata variation, much in the way that stratiﬁed random sampling improves on simple random sampling.
The average price in each matched cell could then be used for the price comparisons using equations (7.32a) and (7.34a), except that – to ensure that the quality diﬀerences in each cell from characteristics other than these major ones did not inﬂuence the price comparison – adjustments were made for quality changes using equations (7.32b) and (7.34b).
This allowed all matched, old unmatched and new unmatched data to be included since, if the average price in, say, a cell of equation (7.32a) was increased because of the inclusion of a new improved item, equation (7.32b) would be used to remove such improvements, on average.

Consider, for example, a brand X, 14-inch television set with stereo sound sold to multiple outlets.

There might be matched cells for brand X television sets sold in multiples, but not matched cells also including stereo.
The new model may have to be grouped in a cell with the brand X, 14-inch television sets sold in multiples, and the average price of the cells compared in equation (7.32a) or (7.34a), and making a quality adjustment for the stereo in the form of equation (7.32b) or (7.34b).

The estimated coeﬃcient for stereo would be derived from a hedonic equation esti- mated from data of other television sets, some of which possess stereo.
increases, (7.32a) and (7.34a) 7.148 The above description illustrates how weighted index number formulae such as Laspeyres, Paasche, Fisher and To¨ rnqvist might be constructed using data on prices, quantities and characteristics of an item.
Silver and Heravi (2003) show that as the number of char- acteristics over which the summation takes place in equations the more redundant the adjustment in equations (7.32b) and (7.34b) becomes, until, when all combinations of char- acteristics are used in equations (7.32a) and (7.34a) as strata, the calculation extends to a matched models problem in which each cell uniquely identiﬁes an item.
For matched data, equations (7.32b) and (7.34b) serve no purpose; the aggregation in equations (7.32a) and (7.34a) would be over all items, and would reduce to the usual index number problem.

Diewert (2003a), com- menting on the method, explains why, when matching is relatively large, the results given are similar to those from superlative hedonic index numbers.
7.149 Weighted index number formulae might thus be constructed using data on prices, quantities and char- 128 acteristics of an item when the data are not matched.

This is because continuing with matched data may lead to errors from two sources: multiple quality adjustments from items no longer available and their non-comparable replace- ments; and sample selectivity bias from sampling from a replacement universe as opposed to a double universe.
The difference between hedonic indices and matched indices 7.150 In previous sections, the advantages of hedo- nic indices over matched comparisons are referred to in terms of the inclusion by the former of unmatched data.

This relationship is discussed more formally here.

Triplett (2002) argued and Diewert (2002e) showed that an unweighted geometric mean (Jevons) index for matched data gives the same result as a logarithmic hedonic index run on the same data.
Consider the mat- ched sample m and Zt+2 and Zt as overall quality adjustments to the dummy variables for time in equation (7.29), that is, PK k=2bkzki.

The very ﬁrst line in equation (7.35) below is shown by Aizcorbe et al.

(2001) to equal the diﬀerence between two geometric means of quality- adjusted prices.

The sample space m=Mt=Mt+2 is the same model in each period.
Consider the introduction of a new model n introduced in period t+2 with no coun- terpart in t and the demise of an old model o so it has no counterpart in t+2.
So Mt+2 is composed of m and n, and Mt is composed of m and o, while M consists only of the matched models m. Silver and Heravi (2002) have shown the dummy variable hedonic comparison to now be: ln pt+2=pt= m=(m+n)P m ln ( pt+2 m Zm)=m +n=(m+n)P n m=(m+o)P m +o=(m+o)P o = m=(m+n)P m m=(m+o)P m n=(m+n)P n o=(m+o)P o ln ( pt+2 n Zn)=n ln ( pt ln ( pt m Zm)=m o Zo)=o ln ( pt+2 m Zm)=m m Zm)=m ln ( pt ln ( pt+2 n Zn)=n ln ( pt o Zo)=o (7.35) 7.151 Consider the second expression in equation (7.35).
First, there is the change for the m matched observations.

This is the change in mean prices of matched models m in period t+2 and t, adjusted for quality.

Note that the weight in period t+2 for this matched compo- nent is the proportion of matched to all observations in period t+2.

Similarly, for period t, the matched weight depends on how many unmatched old observations are in the sample.
In the last line of equation (7.35), the change is between the unmatched new and the un- matched old mean (quality-adjusted) prices in periods t+2 and t. Thus matched methods can be seen to ignore the last line in equation (7.35) and will thus diﬀer from the hedonic dummy variable approach in at least this respect.
The hedonic dummy variable approach, in its inclusion of unmatched old and new observations, can be seen from equation (7.35) possibly to diﬀer from a geo- metric mean of matched prices changes, the extent of any diﬀerence depending, in this unweighted formulation, on the proportions of old and new items leaving and entering the sample and on the price changes of old and new items relative to those of matched ones.
If the market for products is one in which old quality-adjusted prices are unusually low while new quality-adjusted prices are unusually high, then the matched index will understate price changes (see Silver and Heravi, 2002 and Berndt et al., 2003 for examples).

Diﬀerent market behaviour will lead to diﬀerent forms of bias.

7.152 then diﬀerent If sales weights replace the number of obser- vations in equation (7.35), forms of weighted hedonic indices can be derived, as explained in Chapter 21.
Silver (2002) has also shown that the hedonic approach will diﬀer from a corresponding weighted or unweighted hedonic regression in respect of the leverage and inﬂuence that the hedonic regression gives to observations.

Chaining 7.153 An alternative approach to dealing with pro- ducts with a high turnover of items is to use a chained, say monthly, index instead of the long-term ﬁxed base comparison.
A chained index compares prices of items in period t with period t+1 (Indext,t+1) and then, as a new exercise, studies the universe of items in period t+1 and matches them with items in period t+2.

These links (Indext,t+1 and Indext,t+2) are combined by successive multiplication, continuing to, say, Indext+5,t+6 to form Indext+1,t+6.

Only items available in both period t and period t+6 would be used in a ﬁxed base consumer price index.
Consider the ﬁve items 1, 2, 5, 6 and 8 over the four months January–April, as shown in Table 7.1.

The price index for January compared with February (J:F) involves price comparisons for all ﬁve items.

For February–March (F:M) it involves items 1, 4, 5 and 8 and for March–April (M:A) six items: 1, 3, 4, 5, 7 and 8.

The sample composition changes for each comparison as old items disappear and new items come in.
Price indices can be calculated for each of these successive price comparisons using any of the unweighted formulae described in Chapter 21.

The sample will grow in size when new products appear and shrink when old products disappear, changing in composition through time (Turvey, 1999).

7.154 Sample depletion may be reduced in long-run comparisons by the judicious use of replacement items.
As discussed in Chapter 8, however, the replacement sample would only include a new item as and when a replacement was needed, irrespective of the number of new items entering the market.

Furthermore, the re- placement item is likely to be either of a similar quality, to facilitate quality adjustment, and thus have relatively low sales, or of a diﬀerent quality with relatively high ADJUSTING FOR QUALITY CHANGE sales, but requiring an extensive quality adjustment.

In either case this is unsatisfactory.


7.155 Chaining, unlike hedonic indices, does not use all the price information in the comparison for each link.

Items 2 and 6, for example, may be missing in March.

The index makes use of the price information on items 2 and 6 when they exist, for the January–February com- parison, but does not allow their absence to disrupt the index for the February–March comparison.

It may be that item 4 is a replacement for item 2.


Note how easily it is included as soon as two price quotes become available.

There is no need to wait for rebasing or sample rotation.

It may be that item 7 is a replacement for item 6.

A quality adjustment to prices may be required for the February–March comparison between items 6 and 7, but this is a short-run one-oﬀ adjustment, the compilation of the index continuing in March–April using item 7 instead of item 6.

SNA 1993 (Chapter 16, para.


54) on price and volume measurement picks up on the point: In a time series context, the overlap between the pro- ducts available in the two periods is almost bound to be greatest for consecutive time periods (except for sub- annual data subject to seasonal ﬂuctuations).
The amount of price and quantity information that can be utilized directly for the construction of the price or volume indices is, therefore, likely to be maximized by compiling chain indices linking adjacent time periods.

Conversely, the further apart the two time periods are, the smaller the overlap between the ranges of products available in the two periods is likely to be, and the more necessary it becomes to resort to implicit methods of price compar- isons based on assumptions.
Thus, the diﬃculties created by the large spread between the direct Laspeyres and Paasche indices for time periods that are far apart are compounded by the practical diﬃculties created by the poor overlap between the sets of products available in the two periods.

7.156 The chained approach has been justiﬁed as the natural discrete approximation to a theoretical Divisia index (Forsyth and Fowler, 1981 and Chapter 16).
Reinsdorf (1998) has formally determined the theoretical underpinnings of the index, concluding that in general chained indices will be good approximations to the the- oretical ideal – though they are prone to bias when price changes ‘‘swerve and loop’’, as Szulc (1983) has demonstrated (see also Forsyth and Fowler, 1981 and de Haan and Opperdoes, 1997).

7.157 The dummy variable hedonic index uses all the data in January and March for a price comparison between the two months.
Yet the chained index ignores unmatched successive pairs, as outlined above; but this is preferable to its ﬁxed base equivalent.

The hedonic in predicting from a regression equation, approach, naturally has a conﬁdence interval attached to such predictions.

The width of the interval is dictated by the ﬁt of the equation, the distance of the characteristics from their mean and the number of observations.

Matching, chained or otherwise, does not suﬀer from any prediction error.

Aizcorbe et al.
Aizcorbe et al.

(2001) undertook an extensive and meticulous study of high-technology goods (personal computers and semiconductors) using quarterly data for the period 1993 to 1999.

The results from comparable hedonic and chained indices were remarkably similar 129 CONSUMER PRICE INDEX MANUAL: THEORY AND PRACTICE over the seven years of the study.
For, example, for desktop central processing units (CPUs) the index fell between the seven years from 1993:Q1 to 1999:Q4 by 60.0 per cent (dummy variable hedonic), 59.9 per cent (chained Fisher) and 57.8 per cent (chained geometric mean).

The results diﬀered only for quarters when there was a high turnover of items, and in these cases such diﬀerences could be substantial.
For example, for desk- top CPUs in 1996:Q4 the 38.2 per cent annual fall mea- sured by the dummy variable hedonic method diﬀered from the chained geometric mean index by 17 percentage points.

Thus with little model turnover there is little discrepancy between hedonic and chained matched models methods and, for that matter, ﬁxed base matched indices.

It is only when binary comparisons or links have a high model turnover that diﬀerences arise (see also Silver and Heravi, 2001a and 2003).
7.158 Of course it is possible to make up for missing prices by using partial, patched hedonic estimates, as discussed above.

Dulberger (1989) computed hedonic indices for computer processors and compared the results to those from a matched models approach.

The hedonic dummy variable index fell by about 90 per cent over the period 1972 to 1984, about the same as for the matched models approach where missing prices for new or discontinued items were derived from a hedonic regression.
However, when using a chained matched models approach with no estimates or imputations for missing prices, the index fell by 67 per cent.

It is also possible to combine methods; de Haan (2003) used matched data when available and the time dummy only for unmatched data – his double imputation method.

Long-run and short-run comparisons 7.159 This section describes a useful formulation to aid quality adjustment.
Its innovation arises from a possible concern with the long-run nature of the quality- adjusted price comparisons being undertaken.

In the example in Table 7.1, prices in March were compared with those in January.
Assumptions of similar price changes are required by the imputation method to hold over this period for long-run imputations – something that gives rise to increasing concern when price com- parisons continue over longer periods, between January and October, January and November, January and December, and even subsequently.

To help alleviate such concerns, this section considers a short-run formulation, mentioned in paragraph 7.42.
Consider Table 7.5, which, for simplicity, has a single item A that exists throughout the period, an item B which is permanently missing in April, and a possible replacement item C in April.

Quality adjustment methods in short-run comparisons 7.160 A comparable replacement C may be found.

In the previous example the focus was on the use of the Jevons index at the elementary level, since it is shown in Chapter 20 that this has much to commend it.
The example here uses the Dutot index, the ratio of arith- metic means.

This is not to advocate it, but only to provide an example using a diﬀerent formulation.

The Dutot index also has much to commend it on axiomatic grounds, but fails the commensurability (units of mea- surement) test and should only be used for relatively homogeneous items.

The long-run Dutot index for April compared with January is: PD N P i=1 N P i=1 pApr i =N pJan i =N which is 8/5=1.30, a 30 per cent increase.
Table 7.5 Example of long-run and short-run comparisons Item January February March April May Comparable replacement A B C Total 2 3 5 Explicit adjustment A B C Total 2 3 (6/5)3=3.60 5 Overlap A B C Total Imputation A B Total 2 3 5 2 3 5 2 3 5 2 3 5 2 3 5 2 3 5 Figures in bold are estimated quality-adjusted prices described in the text.
130 2 4 6 2 4 6 2 4 5 6 2 6 8 2 (5/6)6=5 6 8 2 6(4/5)=4.8 6 6.8 2 7 9 June 2 8 10 2 (5/6)7=5.8 7 9 2 (5/6)8=6.67 8 10 2 7 2 8 2.5 4 6.5 3.5 (3.5/2.5)4=5.6 9.1 4 (4/3.5)5.6=6.4 8.4 5 (5/4)6.4=8 13 The short-run equivalent is the product of a long-run index up to the immediately preceding period, and an index for the preceding to the current period, i.e., for period t+4 compared with period t: 2 6 6 6 4 PD N P i=1 N P i=1 pt+3 i =N pt i=N 3 7 7 7 5 2 6 6 6 4 N P i=1 N P i=1 3 7 7 7 5 pt+4 i =N pt+3 i =N or for January with April: PD 2 6 6 6 4 N P i=1 N P i=1 pMar i =N p jan i =N 3 7 7 7 5 2 6 6 6 4 N P i=1 N P i=1 3 7 7 7 5 pApr i =N pMar i =N (7.36) 6 5 8 6 which is, of course, =1:30 as before.
7.161 Consider a non-comparable replacement with an explicit quality adjustment.

Say, for example, that C’s value of 6 in April is quality-adjusted to be considered to be worth only 5 when compared to the quality of B.

The quality adjustment to prices may have arisen from an option cost estimate, a quantity adjustment, a sub- jective estimate or a hedonic coeﬃcient, as outlined the long-run comparison uses an above.
Say that adjusted January price for C, which is B’s price of 3 multiplied by 6/5 to upgrade it to the quality of C, i.e., (6/5) 3=3.6.

From April onwards, the prices of the replacement item C can be readily compared to its January reference period price.
Alternatively, the prices of C in April onwards might have been adjusted by multiplying them by 5/6 to downgrade them to the quality of B and enable comparisons to take place with item B’s price in January: for April the adjusted price is (5/6) 6=5; for May the adjusted price is 5.8 and for June it is 6.67 (see Table 7.5).

Both procedures yield the same results for long-run price comparisons.

The results from both methods (rounding errors aside) are the same for item B.
7.162 For the overall Dutot index, however, the results will diﬀer, since the Dutot index weights price changes by their price in the initial period as a proportion of total price (see Chapter 20, footnote 27).

The two quality adjustment methods will have the same price changes, but diﬀerent implicit weights.

The Dutot index in May is 9/5.6=1.607 if an adjustment is made to the initial (January) price or 7.8/5=1.56 if an adjustment is made to the current period (May) price.
The short-run indices give the same results for each adjustment: =1:607 using an adjustment to the initial 9 8 8 5:6 (January) price, and =1:56 using an adjustment to the 7:8 7 7 5 current period (May) price.

ADJUSTING FOR QUALITY CHANGE 7.163 The overlap method may also take the short- run form.

In Table 7.5 there is a price for C in March of 5 that overlaps with B in March.

The ratio of these prices is an estimate of their quality diﬀerence.
A long- run comparison between January and April would be 6 +2 =5=1:36.

The short-run comparison would 4 5 6:8 6 =1:36.
March to April link: be based on the product of the January to March and 6 5 7.164 At this unweighted level of aggregation it can be seen that there is no diﬀerence between the long-run and short-run results when items do not go missing, when comparable replacements are available, when explicit adjustments are made for quality or when the overlap method is used.
The separation of short-run (most recent month-on-month) and long-run changes may have advantages for quality assurance to help spot unusual short-run price changes.

But this is not the concern of this chapter.

The short-run approach does, however, have advantages when imputations are made.
Implicit short-run comparisons using imputations 7.165 The use of the short-run framework has been mainly considered for temporarily missing values, as outlined by Armknecht and Maitland-Smith (1999) and Feenstra and Diewert (2001).

Similar issues nevertheless arise in the context of quality adjustment.

Consider again Table 7.5, but this time there is no replacement item C and item A’s prices have been changed to show an upward trend.

Item B is again missing in April.


A long- run imputation for item B in April is given by (3.5/2) 3=5.25.

The price change is thus (5.25+3.5)/5=1.75 or 75 per cent.

This is, of course, the same result as that obtained by simply using item A (3.5/2=1.75), since the implicit assumption is that price movements of item B, had it continued to exist, would have followed those of A.

The assumption of similar long-run price movements may, in some instances, be diﬃcult to support over very long periods.
An alternative approach would be to use a short-run framework in which the imputed price for April is based on the (say, overall) mean price change between the preceding and current period, i.e.

(3.5/ 2.5) 4=5.6 in the above example.

In this case, the price change between March and April is (5.6+3.5)/ (2.5+4)=1.40.

This is combined with the price change between January and March 6.5/5=1.30, to give the January to April change of 1.30 1.40 =1.82, an 82 per cent increase.
7.166 Consider why the short-run result of 82 per cent is larger than the long-run result of 75 per cent.

The price change for A between March and April of 40 per cent, upon which the short-run imputation is based, is larger than the average annual change of A, which is just over 20 per cent.
The extent of any bias from this approach was found, above, to depend on the ratio of missing values, and the diﬀerence between the average price change of the matched sample and the quality- adjusted price change of the item that went missing, had it continued to exist.

The short-run comparison is to be 131 CONSUMER PRICE INDEX MANUAL: THEORY AND PRACTICE favoured if the assumption of similar price changes is considered more likely to hold than the long-run assumption.
7.167 There are data on price changes of the item that are no longer available, item B in Table 7.5, up to the period preceding the period in which it is missing.

In Table 7.5, item B has price data for January, February and March.

The long-run imputation makes no use of such data, simply assuming that price changes over the period of January to April, for example, are the same for B as for A.
Let the data for B’s prices in Table 7.5 (penultimate row) now be 3, 4 and 6 in January, Feb- ruary and March, respectively, instead of 3, 3 and 4.

The long-run estimate for B in April is 5.25, as before.

The estimated price change between March and April for B is now a fall from 6 to 5.25.

A short-run imputation based on the price movements of A between March and April would more correctly show an increase from 6 to (3.5/ 2.5) 6=8.4.
7.168 There may, however, be a problem with the continued use of short-run imputations.

Returning to the data for A and B in Table 7.5, consider what happens in May.

Adopting the same short-run procedure, the imputed price change is given in Table 7.5 as (4/3.5) 5.6=6.4 and for June as (5/4) 6.4=8.
In the former case, the January to May price change is: (6:4+4) (5:6+3:5) (5:6+3:5) (3+2) =2:08 and in the latter, for June: (8+5) (6:4+4) (6:4+4) (3+2) =2:60 compared with long-run comparisons for May and June, respectively, of: ((4=2) 3+4) (3+2) ((5=2) 3+5) (3+2) =2:00 =2:50 7.169 A note of caution is required.
The compar- isons here use an imputed value for item B in April and also an imputed value for May.

The price comparison for the second term in equation (7.36) above, for the current versus immediately preceding period, uses imputed values for item B.

Similarly, for the January to June results, the May to June comparison uses imputed values for item B for both May and June.

The pragmatic needs of quality adjustment may of course demand this.
If comparable replacements, overlap links and resources for explicit quality adjustment are unavailable, an imputation must be considered.

However, using impu- ted values as lagged values in short-run comparisons introduces a level of error into the index which will be compounded with their continued use.
Long-run imputations are likely to be preferable to short-run changes based on lagged imputed values, unless there is something in the nature of the industry that cau- tions against such long-run imputations.
There are 132 circumstances in which the price collector may believe the missing item to be temporarily missing, and the imputation is conducted in the expectation that pro- duction will subsequently continue; a wait-and-see pol- icy is adopted under some rule, say that the item is missing for a maximum of three months, after which the item is deemed to be permanently missing.
Such prag- matic situations require imputations to extend values over consecutive periods and call for the use of lagged imputed values to be compared with current imputed values, despite the fact that this is cautioned against, especially over several months.

There is an intuitive feeling that the period over which this is undertaken should not be extensive.

First, the eﬀective sample size decreases as the use of imputation increases.
Second, the implicit assumptions of similar price movements inher- ent in imputations are less likely to hold over the longer run.

Finally, there is some empirical evidence, albeit from a diﬀerent context, against the use of imputed values as if they were lagged actual values (see Feenstra and Diewert’s study (2001) using data from the United States Bureau of Labor Statistics for their International Price Program).
7.170 The above short-run approach will be devel- oped in the next section, where weighted indices are considered.

The practice of estimating quality-adjusted prices is usually carried out at the elementary item level.

At this lower level, the prices of items may sub- sequently be missing, and replacements with or without adjustments and imputations are used to allow the series to continue.

New items are also being introduced, as are newer varieties and switching of sales between sections of the index.
The turmoil of changing quality is not just about maintaining similar price compari- sons, but also about the accurate reweighting of the mix of what is consumed.

Under a Laspeyres framework, the bundle is held constant in the base period, so any change in the relative importance of items consumed is held to be of no concern until the next re-basing of the index.

Yet procedures for updating the weights are required to capture something of the very real changes in the mix of what is consumed.
This is considered in Chapter 9.

The concern here is with an equivalent higher-level procedure to the short-run adjustments discussed above.

It is one particularly suited to coun- tries where resource constraints prohibit the regular updating of weights through regular household surveys.

Single-stage and two-stage indices 7.171 Consider aggregation at the elementary level.
This is the level at which prices are collected from a representative selection of outlets across regions in a period and compared with the matched prices of the same items in a subsequent period to form an index for, say, lamb.

Each price comparison is equally weighted unless the sample design gives proportionately more chance of selection to items with more sales.
The ele- mentary price index for lamb is then weighted, and combined with the weighted elementary indices for other products to form the consumer price index.
A Jevons elementary aggregate index for period t+6 compared with period t, for example, is given as: PJ N Q ( pt+6 i i 2 N(t+6)\N(t) =pT i ): (7.37) Compare this with a two-stage procedure: PJ N Q ( pt+5 i i 2 N(t+5)\N(t) =pT i ) N Q ( pt+6 i i 2 N(t+6)\N(t+5) =pT+5 i ) (7.38) 7.172 If an item is missing in period t+6, an impu- tation may be undertaken.
If equation (7.37) is used, the requisite assumption is that the price change of the miss- ing item, had it continued, is equal to that of the average of the remaining items over the period t to t+6.

In equation (7.38), the missing item in period t+6 may be included in the ﬁrst stage of the calculation, between periods t and t+5, but excluded in the second stage, between periods t+5 and t+6.

The requisite assumption is that price changes between t+5 and t+6 are equal.
Assumptions of short-run price changes are generally considered to be more valid than their long-run counter- parts.

The two-stage framework also has the advantage of including in the worksheet prices for the current period and the immediately preceding one which, as is shown in Chapter 9, promotes good data validity checks.
7.173 Feenstra and Diewert (2001) applied a num- ber of mainly short-run imputation procedures to ADJUSTING FOR QUALITY CHANGE price comparisons for the United States Bureau of Labor Statistics International Price Program.

Although such price indices are not the direct interest of this manual, the fact that about one-quarter of the indivi- dual items tracked did not have price quotations in any given month makes it an interesting area in which to explore the results from diﬀerent imputation pro- cedures.
When using the two-stage procedure, Feenstra and Diewert (2001) advise against carrying forward imputed period prices as if they were actual values, for the subsequent price comparison.

The resulting price relatives for the subsequent period based on prior imputations had a standard deviation about twice that of price relatives where no imputation was required, leading the authors to conclude that such a practice introduced a signiﬁcant amount of error into the cal- culation.
Feenstra and Diewert (2001) found that higher variances of price changes arose from long-run impu- tation compared with the short-run imputation method.

They also found, from both theory and empirical work, that when actual prices become available in a future data set and were used to interpolate back on a linear basis the missing prices, then such estimates lead to much lower variances than the short-run imputation approach.
Such linear interpolations, however, require the statistical agency to store past information until a price quote becomes available, interpolate back the missing price, and then publish a revised consumer price index.

133 CONSUMER PRICE INDEX MANUAL: THEORY AND PRACTICE Appendix 7.1 Data on personal computers, obtained from United Kingdom Compaq and Dell web sites, July 2000, to illustrate hedonic regression PRICE (£) SPEED (MHz) RAM, MB.

HD, MB.
DELL PRESARIO PROSIGNIA CELERON PENTIUM III CD-RW DVD DELL*SPEED (MHz) ADJUSTING FOR QUALITY CHANGE 1609 1389 999 1119 1099
FOR QUALITY CHANGE 0
667 667 667 566 566 566 566 566 566 566 733 733 733 733 733 933 933 933 933 933 135 ITEM SUBSTITUTION, SAMPLE SPACE AND NEW PRODUCTS 8 Introduction 8.1 As new items are introduced and old items no longer sold, the universe of items from which prices are sampled changes.
Yet index number methodology may constrain the sampling to subsets of the universe.

The samples selected from such subsets are referred to here as the ‘‘sample space’’ of the index.

A focus of this chapter is the limitations of such sample spaces.

In Chapter 7 the use of the matched models method was recognized as the accepted approach to ensuring that the measurement of price changes was untainted by changes in quality.
It was noted, however, that the approach might fail in three respects: missing items, the limited sample space, and new goods and services (in the remainder of this chapter ‘‘goods’’ is taken to include services).

In Chapter 7, several implicit and explicit methods of quality adjustment to prices, and the choice between them, are discussed as ways of dealing with missing items.
In this chapter, attention is turned to the two other reasons why the matched models method may fail: sampling concerns (the limited sample space) and new products.

The three sources of potential error are ﬁrst brieﬂy outlined below.

8.2 Missing items.

A problem arises when an item is no longer produced.
An implicit quality adjustment may be made using the overlap or imputation method, or the respondent may choose a replacement item of a com- parable quality and its price may be directly compared with the missing item’s price.

If the replacement is of a non-comparable quality, an explicit price adjustment is required.

This was the subject of Chapter 7, para- graphs 7.72 to 7.115.

In paragraphs 7.125 to 7.158 a caveat was added.


It was recognized that for items in industries where model replacements are rapid, con- tinued long-run matching depletes the sample and quality adjustment becomes unfeasible on the scale required.

Chained matching or hedonic indices are deemed pre- ferable.

8.3 Sampling concerns.


The matching of prices of identical items over time, by its very nature, is likely to lead to the monitoring of a sample of items that is in- creasingly unrepresentative of the population of trans- actions.

Price collectors may keep following those selected items until they are no longer available.

Thus, price collectors may continue to monitor old items, with unusual price changes and limited sales.
With regard to item replacement, price collectors may select unpopular comparable items in order to avoid explicit quality adjust- ments.

Thus obsolete items with unusual price changes may be replaced by near obsolete items, again with unusual price changes.

That the replacement items are near obsolete will mean that their expenditure shares will be relatively small.

This will compound the problem of unrepresentative samples.
The substitution of an item with relatively high sales for an obsolete one has its own problems, since the diﬀerence in quality is likely to be substantial and substantive, beyond that which can be attributed to, say, the price diﬀerence in some overlap period.

One item might be in the last stage of its life cycle and the other in the ﬁrst stage of its life cycle.

The prob- lem has implications for sample rotation and item sub- stitution.

8.4 New products.
8.4 New products.

A third potential diﬃculty arises when something ‘‘new’’ is produced.

There is a diﬃculty in distinguishing between new items and quality changes in old ones, and this is discussed below.

When a quite new good is produced, there is a need for it to be included in the index as soon as possible, especially if the product is expected to be responsible for relatively high sales.
New goods might have quite diﬀerent price changes from those of existing ones, especially at the start of the life cycle.

Furthermore, in the initial period of introduction there is often a welfare gain to the consumer.

The new good is not a perfect substitute for the old good and this uniqueness gives economic value to the consumer which would have otherwise not been obtained, had the new good not been avail- able (Trajtenberg, 1989).
But by deﬁnition, there is no price for the new product in the period preceding its introduction.

So even if prices of new products are obtained and included in the index from the initial introduction date, there is still something missing – the initial gain in welfare that consumers experience in the period of introduction.

The diﬃculties in capturing such eﬀects are discussed in paragraphs 8.59–8.60 and Appendix 8.2.

8.5 The problem of missing items was the subject of Chapter 7.
This chapter considers sampling concerns arising out of the matched models approach and the problem of introducing new products into the index.

Matched samples 8.6 The matching procedure has at its roots a conundrum.

Matching is designed to avoid price changes being contaminated by quality changes.

Yet its adoption constrains the sampling to a static universe of items that exist in both the reference and base periods.
Outside this matched sample, there are of course the items that exist in the reference period but in the current period, and are therefore not not matched, and similarly those new items existing in 137 CONSUMER PRICE INDEX MANUAL: THEORY AND PRACTICE the current period but not in the reference period – the dynamic universe (Dale´ n, 1998a; Sellwood, 2001).
The conundrum is that the items not in the matched uni- verse – the new items appearing after the reference period and the old items that disappeared from the current period – may experience price changes that diﬀer substantially from the price changes of existing matched items.

This is because these products will embody diﬀerent technologies and be subject to dif- ferent (quality-adjusted) strategic price changes.
The very device used to maintain a sample of constant quality, i.e., matching, may itself give rise to a sample that is biased away from technological developments.

Furthermore, when this matched sample is used to impute the price changes of missing items (see Chapter 7, paragraphs 7.53 to 7.68), it will reﬂect the technology of a sample that is not representative of current tech- nological changes.
8.7 A formal consideration of matching and the dynamic universe is provided in Appendix 8.1 to this chapter.
Three universes are considered: an intersection universe, which includes only matched items; a dynamic double universe, which includes all items in the base comparison period and all in the current period, although they may be of diﬀerent qualities; a replacement universe, which starts with the base period universe, but also includes one-to-one replace- ments when an item from the sample in the base period is missing in the current period.
8.8 It is, of course, diﬃcult to ascertain the extent to which matching from the intersection universe con- strains the penetration of the sample into the dynamic, double universe, since statistical agencies generally do not collect data for the latter.

Its extent will in any event vary between products.

Sellwood (2001) advo- cated simulations using the universe of scanner data.
Silver and Heravi (2002) undertook such an experiment using scanner data on the consumer prices of washing machines in the United Kingdom in 1998.

A matched Laspeyres index, based on price comparisons with matched models existing in both January and Decem- ber, covered only 48 per cent of December expenditure on washing machines, as a result of new models intro- duced after January not being included in the matched index.
Furthermore, the January to December matched comparison covered slightly more than 80 per cent of January expenditure, resulting from the exclusion of models available in January but not in December.

A biannual increased the December expenditure coverage to just over 70 per cent, while a monthly (chained) rotation increased that cov- erage to 98 per cent (see also Chapter 7, paragraphs 7.128 to 7.131 for further examples).

Two implications arise from this.


First, the selection of item substitutes (replacements) puts the coverage of the sample to some extent under the control of the price collectors.

Guide- lines on directed replacements in particular product areas have some merit.

Second, chaining, hedonic indi- ces (as considered in Chapter 7, paragraphs 7.125 to 7.158) and regular sample rotation have merit in some sample rotation (rebasing) 138 product areas as devices to refresh the sample.
These are considered in turn.

Sample space and item replacement or substitution 8.9 When an item goes missing, one possibility is for the price collector to select a replacement item.

The sample space of the index is thus the matched items initially selected and replacement items selected when matched items are missing.

Price collectors are often best placed to select replacement items.
The price collectors are often physically present in the same store as the missing item and thus any replacement price selected is likely to be unaﬀected by price diﬀerences which may be attributed to diﬀerences in the services (ease of location, parking, warranties, service) provided by diﬀerent stores.

It may also be the case that an obvious replacement is provided by a store which wishes to cater to the same market segment, and this will be conspicuous to the price collector.
Sometimes the replacement may have a dif- ferent code or model number which a desk oﬃcer may take to indicate a diﬀerent item, but which the price collector can identify as simply being a diﬀerence of, say, a colour or packaging.

Price collectors can also identify whether a new (replacement) model of an item has styling and other qualitative factors so diﬀerent from the old model that in themselves they would account for sub- stantial price diﬀerences.
In such instances, a desk oﬃcer may only focus on the technical speciﬁcations and be unaware of these other diﬀerences.

Against this, desk oﬃcers have additional information.

This might include information from a similar store in a diﬀerent location on the price of the missing item, which might be tem- porarily out of stock.

8.10 The price collector takes on the task of identi- fying whether an item is of comparable quality or not.
If the price collector judges the item to be comparable when in fact it is not, the quality diﬀerence will be taken to be a price diﬀerence, resulting in bias where the unrecognized quality changes are in a consistent direction.

Informed comparable substitution requires general guidelines on what makes a good substitute, as well as product-speciﬁc information on characteristics likely to determine price.
It also requires timely substitution, to maximize the probability of an appropriate substitute being available.

8.11 Guidelines for selection of comparable items and monitoring of the nature of the selections are good practice.

Liegey (1994) notes how useful the results from hedonic regressions are in the selection of items.

The results provide an indication of the major quality factors that explain price variation in the product or service.
Price collectors can thus be given guidelines as to which characteristics are important – in the sense that they are price determining – in the selection of the sample and replacement items.

8.12 The matter of sample space requires considera- tion regarding the selection of replacement items/sub- stitutes for missing items.

The initial selection of items whose prices are matched may best be made at random, though such items are more often selected as those ‘‘typically’’ purchased.
Similarly items ‘‘typically’’ pur- chased should be included as replacements.

Not all price collectors should aim to sample the same ‘‘most typical’’ item.

It is desirable to sample a distribution of items which broadly represents the distribution of purchases.

For example, a particular brand – one that accounts for, say, 40 per cent of sales revenue – may be known to be the market leader.

This common knowledge should not lead all price collectors to select that brand on rebasing.
A representative sample is required.

8.13 Replacement items should intrude into the universe of transactions so that the sample is broadly representative of the dynamic universe.

The inclusion of a popular replacement item to refresh the sample – one at the same point in its life cycle as the original popular one selected in the base period – allows for a useful and accurate price comparison, assuming that an appropriate quality adjustment is made.
Substitute or replacement items should, where possible, not merely be comparable in quality, but should also be likely to account for a relatively substantial amount of sales value.

It is of little merit to substitute a new item with limited sales for a missing item, again with limited sales, just because they have similar features, both being ‘‘old’’; the index would become more unrepresentative.
The replacement of an item only when the item is no longer available may be ineﬀectual with regard to the representativity of the index.

In that case, items with relatively low sales would continue to be monitored until they died.

And even replacement might not remedy the situation.

If the replacement guidelines indicate that the price collector should select a similar item sold in the outlet, then the replacement selected will be almost as obsolete (Lane, 2001, p. 21).
8.14 Guidelines to select ‘‘similar’’ items are given to ease quality adjustment between the old and new items; at best the items are ‘‘comparable’’ and require no quality adjustment.

The institutional mechanism devised to help in making quality adjustments to prices can lead to bias because of its adherence to a sample of items which do not enjoy the beneﬁts of recent technological innovations and are unrepresentative of what is pro- duced.
Bear in mind that an index number methodology based on an initially selected matched sample and a sample of substitute replacement items, when items go missing, may not be representative of the universe of all items being consumed.

In particular, if the index number methodology is biased to the selection of replacement items with relatively low sales, so that they are compar- able with obsolete items, then the sampling from new items and the sample space of the index are biased.
Quality adjustment and representativity are interrelated, since the former aﬀects the sample space of the index.

8.15 The importance of, and care required in, the use of replacements to militate against sample depletion is worth reiterating.

Consider the case where there is only one model of a product available in the market at the start of the price comparison in period t. A price collector includes it in the sample in period t and then monitors its price in subsequent periods.
A new (replacement) model enters the market in, say, period t+2, but it is ignored since the original model continues to exist for several ITEM SUBSTITUTION, SAMPLE SPACE AND NEW PRODUCTS months.

However, in, say, period t+9 the old item is no longer on the market and is replaced, with a quality adjustment, by the new item.

The long-run price com- parison between the new model’s price in period t+9 and the old model’s price in period t has no sampling bias.
Both account for 100 per cent of the market in their respective periods, being the only items available.

Both are near the start of their life cycles, so the price com- parison is a fair one.

If the new and old items have dif- ferent price changes, sampling bias will occur between periods t+2 and t+8, when only one of the two items is being sampled, but sampling will be unbiased once the model is replaced in period t+9.
8.16 There is thus a case for managing the replace- ment strategy to minimize sample depletion.

In that respect, the following points should be borne in mind: Replacements oﬀer an opportunity to cut back on, and possibly remove, sample bias in the period of replace- ment, though not prior to it.

 The more frequent is the replacement, the less the sample bias.
If there is more than one new (replacement) item in the market, there may still be bias as only the most pop- ular one will be selected and it may well be at a dif- ferent stage in its life cycle and thus be experiencing diﬀerent price changes in comparison with other new (replacement) models.

 The analysis assumes that perfect quality adjustments are made on replacement.
The less frequent the replace- ment, the more diﬃcult this might be to achieve, as the very latest replacement item on the market may have more substantial diﬀerences in quality than earlier ones.

 If the best-selling replacement item is of comparable quality and at the same stage in its life cycle as the missing item, then its selection will minimize sample bias.
If there is more than one replacement item and the most comparable one – having the old technology – is selected, it will have low market share and unusual price changes.

 Given advance information on market conditions, replacements that are included in the sample well before the old item becomes obsolescent are likely to increase the sample’s share of the market, include items that are more representative of the market, and facilitate quality adjustment.
8.17 The problem of item substitution is analogous to the problems that arise when an outlet closes.

It may be possible to ﬁnd a comparable outlet not already in the sample, or a non-comparable one for which, in principle, an adjustment can be made for the better quality of service provided.

It is not unusual for an outlet to close following the introduction of a new, more competitive outlet.
Where the matching of prices between these outlets broadly follows the consumption patterns of the clients of the original outlet, there is an obvious replace- ment outlet.

If, however, the new outlet has comparable prices but, say, a better range of items, parking and service, there is a gain to consumers from substituting 139 CONSUMER PRICE INDEX MANUAL: THEORY AND PRACTICE one outlet for the other.
Yet since such facilities have no direct price, it is diﬃcult to provide estimates of their value in order to make an adjustment for the better quality of service of the new outlet.

The index would thus have an upward bias, which would be lost on rebasing.

In such cases, replacing the old outlet by a new one that provides a similar standard of service may be preferable to replacing it by one that has a diﬀerent standard but serves the same catchment area.
In their regression analyses for consumer durables, Liegey (2000), Shepler (2000) and Silver and Heravi (2001b) found ‘‘outlet type’’ to be a substantial and statistically signiﬁcant explana- tory variable for price variation, while for a particular outlet type – grocery outlets, for food and petrol prices in the United States – Reinsdorf (1993) found much smaller diﬀerences.
Sample rotation, chaining and hedonic indices 8.18 It is important to recognize the interrelation- ships among the methods for handling item rotation, item replacement and quality adjustment.

When con- sumer price index (CPI) item samples are rotated, this is a form of item substitution, except that it is not ‘‘forced’’ by a missing item, but is undertaken for a general group of items to update the sample of items.

It has the eﬀect of making future forced replacements less likely.
Yet the assumption implicit in its use is equivalent to that for the overlap adjustment technique: that price diﬀerences are an adequate proxy for the change in price per unit of quality between items disappearing from the sample and replacement items.

8.19 Consider the initiation of a new sample of items.

This may be by probability or judgemental methods, or a combination of the two.
Prices for the old and new samples are returned in the same month, and the new index is compiled on the basis of the new sample, the results being linked to the old.

This is an implicit use of the overlap method, in which all price diﬀerences between the new and old items in that month are taken to be quality changes.

Assume that the new sample is initiated, say, in January.
Assume also that the prices of an old item in December and January are $10 and $11 respectively, a 10 per cent increase, while those for the new replacement item in January and February are $16 and $18 respectively, an increase of 12.5 per cent.

The new item in January is of a better quality than the old, and this diﬀerence in quality may be worth $1611=5 to the consumer.
That is, the price diﬀerence is assumed to be equal to the quality diﬀerence, which is the assumption implicit in the overlap method.

Had the price of the old item in December been compared with the quality-adjusted price of the new item in January under this assumption, the price change would in this case be the same, 10 per cent (i.e.

(165)/10=1.10).
(165)/10=1.10).

In practice, the need to simultaneously replace and update a large number of items requires the assumptions of the overlap method, the point being that this process should not be regarded as error free.

In cases where the assumptions are considered likely to be particularly 140 untenable (discussed in Chapter 7, paragraphs 7.44 to 7.52), explicit adjustments of the form discussed in paragraphs 7.72 to 7.115 should be used.
8.20 It was noted above that when samples are updated, any diﬀerence in the average quality of items between the samples is dealt with in a way that is equivalent to the overlap adjustment technique.

Sample rotation to refresh the sample between rebasing is an expensive exercise.

If rebasing is infrequent, however, and if there is a substantial loss of items in particular product areas, then sample rotation might be appro- priate for those areas.
A metadata system (described below) will aid such decision-making.

More frequent sample rotation aids the process of quality adjustment in two ways.

First, the new sample will include newer vari- eties.

Comparable replacements with substantial sales will be more likely to be available and non-comparable ones will be of a similar quality, which facilitates good explicit adjustments.
Second, because the sample has been rotated, there will be fewer missing items than otherwise and thus less need for quality adjustments.

8.21 A natural extension of more frequent sample rotation is to use a chained formulation in which the sample is reselected each period.

In Chapter 7, para- graphs 7.153 to 7.158, the principles and methods were outlined in the context of sectors in which there was a rapid turnover of items.

These principles are echoed here.


Similarly, the use of hedonic indices (as outlined in paragraphs 7.132 to 7.152) or the use of short-run comparisons (discussed in paragraphs 7.159 to 7.173) might be useful in this context.

Information requirements for a quality adjustment strategy 8.22 It should be apparent from the above that a strategy for quality adjustment must not only be linked to one relating to sample representativity, but must also require the building of a statistical metadata system.
This is not an area where the approach for the index as a whole can be simply described, but one that requires the continual development of market information and the recording and evaluation of methods on a product- by-product basis.

Statistical metadata system 8.23 The methods used for estimating quality- adjusted prices should be well documented as part of a statistical metadata system.
Metadata are systematic descriptive information about data content and organi- zation that help those who operate the systems that produce statistics to remember what tasks they should perform and how they should perform them.

A related purpose is to train new staﬀ and introduce them to the production routines (Sundgren, 1993).

Metadata systems also help to identify where current methods of quality adjustment require reconsideration, and prompt the use of alternative methods.
They may also serve user needs, the oldest and most extensive form being footnotes.

8.24 The dramatic increase in volume of statistical data in machine-readable form, with a concomitant increase in metadata, argues for keeping the metadata in such a form.

This is to enhance transparency in the methods used and help ensure that the methods are understood and continued, as staﬀ leave the CPI team and new staﬀ join it.

Changes in quality adjustment methodology can in themselves lead to changes in the index.
Indices produced using new procedures should be spliced onto existing indices.

The metadata system should also be used as a tool to help with quality adjust- ment.

Because so much of the rationale for the employ- ment of diﬀerent methods is speciﬁc to the features of the products concerned, data should be held on such features.

8.25 Statistical agencies should monitor the incidence of missing items against each Classiﬁcation of Individual Consumption according to Purpose (COICOP) group.
If that incidence is high, then the monitoring should be carried out by class within each group.

Again, if that incidence is high, the monitoring should be done by ele- mentary aggregate or selected representative items within each group, or at the most detailed level of the system.
Where the incidence is high, the ratios of temporary missing prices, comparable replacements and non-com- parable replacements to the overall number of prices, and the methods for dealing with each of these three cir- cumstances, should also be monitored to provide the basis of a statistical metadata system.

The advantage of a top-down approach is that resources are saved by moni- toring only at the detailed level product areas which are problematic.
such as information, 8.26 Product-speciﬁc the timing of the introduction of new models, pricing poli- cies (especially with regard to months in which no changes were made) and the popularity of models and brands according to diﬀerent data sources, should be included in the metadata as the system develops.

An estimate, if available, of the weight of the product concerned should be given, so that a disproportionate eﬀort is not given to relatively low-weighted items.
All this will lead to increased transparency in the procedures used and allow eﬀort to be directed where it is most needed.

8.27 For items for which replacement levels are high, the metadata system would beneﬁt from contacts be- tween statistical agencies and market research organiza- tions, retailers, manufacturers and trade associations.

Such links will allow staﬀ to better judge the validity of the assumptions underlying implicit quality adjustments.
Where possible, staﬀ should be encouraged to be respon- sible for learning more about speciﬁc industries whose weights are relatively high and where item replacement is common.

8.28 Statistical staﬀ should identify price-determining characteristics for product areas using hedonic regres- sions, information from market research, store man- agers, trade associations and other such bodies, and the experience of price collectors.
This information should contribute to the statistical metadata system and be particularly useful in providing subsequent guidelines on item selection.
8.29 When hedonic regressions are used either for partial patching of missing prices or as indices in their ITEM SUBSTITUTION, SAMPLE SPACE AND NEW PRODUCTS own right, information on the speciﬁcation, estimated parameters and diagnostic tests of the regression equa- tions should be kept, along with the data and with notes as to why the ﬁnal formulation was chosen and used.

This will allow the methodology for subsequent updated equations to be benchmarked and tested against the previous versions.
8.30 The metadata system should help statistical staﬀ to: – identify product areas likely to be undergoing regular technological change; – ascertain the pace at which models change and, pos- sibly, the timing of changes; – undertake an analysis of what have in the past been judged to be ‘‘comparable’’ replacements in terms of the factors that distinguish the replacement and old items; – identify whether diﬀerent price collectors are making similar judgements regarding comparable replace- ments, and whether such judgements are reasonable.
8.31 Price statisticians may have more faith in the use of some quality adjustment procedures than others.

When such procedures are used extensively it might be useful to note, as part of the metadata system, the degree of faith the statistician has in the procedures.
Follow- ing Shapiro and Wilcox (1997b), this may be envisaged as a traditional conﬁdence interval: the statistician may believe at, say, a 90 per cent level of conﬁdence that the quality-adjusted price change is 2 per cent (0.02) with an interval of plus or minus 0.5 per cent (0.005).

There may be an indication as to whether the interval is symmetric, or positively or negatively one-sided.

Alternatively, sta- tisticians may use a simple subjective coding on a scale of, say, 1 to 5.
New products and how they differ from products with quality changes 8.32 The question arises of how to deﬁne new prod- ucts (goods and services) and to distinguish them from existing products whose quality has changed.

A new model of a good may provide more of a currently available set of service ﬂows.

For example, a new car model may diﬀer from existing ones in that it may have a bigger engine.
There is a continuation of a service and production ﬂow, and this may be linked to the service ﬂow and production technology of the existing models.

A practical deﬁnition of a new good, as against quality changes in an updated existing model, is that, ﬁrst, the new good cannot easily be linked to an existing item as a continuation of an existing resource base and service ﬂow, because of the very nature of its ‘‘newness’’.
For example, frozen foods, microwave ovens and mobile phones, while extensions of existing ﬂows of services to the consumer, have a dimension of service that is quite new.

Second, as discussed below, new goods can gen- erate a welfare gain to consumers by their very intro- duction.

The simple introduction of the new good into the index, once two successive price quotes are available, misses this gain.
141 CONSUMER PRICE INDEX MANUAL: THEORY AND PRACTICE 8.33 Oi (1997) directs the problem of deﬁning ‘‘new’’ goods to that of deﬁning a monopoly.

If there is no close substitute, the good is new.

He argues that indi- vidual new books, new videos and new television serials may have quite small cross-price elasticities in some cases; their shared service is to provide entertainment and they are similar in this respect.
Hausman (1997), however, found cross-elasticities for substitution to be quite substantial for new television serials (though see Bresnahan (1977)).

There are many new forms of existing products, such as fashionable toys and clothes, which are not easily substitutable for similar items and for which consumers would be willing to pay a premium.
8.34 Bresnahan (1997, p. 237) notes that Brandweek counted over 22,000 new-product introductions for the United States for 1994 – the purpose of their introduc- tion being, as diﬀerentiated products, not to be exact substitutes for existing ones, but to be distinct.

Their distinctiveness is in many cases the rationale behind their launch.

The extent of diﬀerentiated markets nevertheless makes the deﬁnition and treatment of such things as ‘‘new’’ impractical.
Oi (1997, p. 110) sets out the prag- matic case: ‘‘Our theory and statistics would be unduly cluttered if separate product codes had to be set aside for Clear Coke and Special K.’’ Furthermore, the techniques for including such products are not, as indicated below, readily applicable.

The sound practical advice given by Oi (1997) to keep matters ‘‘uncluttered’’ is therefore not unreasonable.
8.35 The terminology adopted here is that used by Merkel (2000) for producer price index (PPI) measure- ment, but considered in a CPI context.

It distinguishes between evolutionary goods and revolutionary goods.

Evolutionary goods are replacement or supplementary models that continue to provide a similar service ﬂow, but maybe in new ways or to diﬀerent degrees.

These are distinguished from revolutionary goods, which are entirely new goods not closely tied to a previously avail- able product.
Although revolutionary goods may satisfy a long-standing consumer need in a novel way, they do not ﬁt into any established CPI item category (Arm- knecht et al., 1997).

Problems are associated with incorporating distinctly new revolutionary goods.

This is because a good, which by its nature is unique, is unlikely to be incorporated into the sample as a replacement for an existing item.
It would neither be comparable nor be amenable to explicit adjustments to its price for quality diﬀerences with existing goods.

Since a distinctly new item is not replacing an item, it does not have an existing weight; its introduction therefore implies a need to re- weight the index.

Incorporation of new products 8.36 There are three major concerns regarding the incorporation of new goods into the CPI.
The ﬁrst con- cern is the detection and identiﬁcation of the new good; these are facilitated by close links with market research, and producer and trade associations.

The second con- cern, which is related to the ﬁrst, is the decision on the need and timing for their inclusion.

This refers to both 142 the weight and price changes of the new good.

The third concern relates to the incorporation of the initial welfare to the consumer arising from the switch from the old technology.
8.37 Consider some examples on the timing of the introduction of new goods.

The sales of mobile phones were at such a signiﬁcant level in some countries that their early inclusion in the CPI became a matter of priority.

They simply rose from nothing to relatively quickly account for quite a large proportion of sales in their product classiﬁcation.

Furthermore, their price changes were atypical of other goods in their product classiﬁcation.
Being new, they might be produced using inputs and technologies quite diﬀerent from those used for existing telephones.

Because of substantial marketing campaigns, many new goods command substantial sales and are the subject of distinct pricing strategies at launch.

For radical innovations, however, there may be a delay in their incorporation into the index since they cannot be deﬁned within existing classiﬁcation systems.

8.38 Armknecht et al.


(1997) cite the example of the incorporation of video cassette recorders (VCRs) into the United States CPI.

VCRs were launched in 1978 with a sales value of US$299 million and estimated average retail price of US$1,240.

Because the CPI was rebased every ten years, VCRs were introduced into the CPI only in 1987 when their sales value was US$3,442 million and average price had fallen to US$486.

All the extraordinary price movements between 1978 and 1987 were thus missed by the index.
8.39 Dulberger (1993) provides some estimates for United States PPIs for dynamic random access memory (DRAM) computer memory chips.

She calculated price indices for the period 1982 to 1988 with varying amounts of delay in introducing new chips into the index.

The indices were chained so that new chips could be intro- duced, or not, as soon as they had been available for two successive years.
Using a Laspeyres chained index, there was a fall of 27 per cent if there was no delay in intro- ducing the new goods, as compared with falls of 26.2 per cent, 24.7 per cent, 19.9 per cent, 7.1 per cent and 1.8 per cent if the introductions were delayed by, respectively, 1, 2, 3, 4 or 5 years.

In all cases the index was biased downwards because of the delay.

Berndt et al.

(1997) provide a detailed study of a new anti-ulcer drug, Tagamet.
They found that the eﬀects of pre-introduction marketing of the drug on its price and market share at introduction were substantial.

Not unexpectedly, there were price falls for the generic form of a pharmaceutical on the expiry of the patent, but there were increases for the branded form; loyal customers were found to be willing to pay a premium over the price prior to the patent expiry (Berndt et al., 2003).
8.40 Waiting for a new good to be established or waiting for the rebasing of an index before incorporating new products may lead to errors in the measurement of price changes if the unusual price movements at critical stages in the product life cycles are ignored.

Strategies are required for the early identiﬁcation of new products, and mechanisms are needed for their incorporation either at launch (if preceded by major marketing strategies) or soon after (if there is evidence of market acceptance).
These strategies and mechanisms should form part of the metadata system.

Waiting for new products to achieve market maturity may result in an implicit policy of ignoring the quite disparate price movements that accompany their introduction (Tellis, 1988, and Parker, 1992).

This is not to say that new goods will always have diﬀerent price changes.

Merkel (2000) gives the example of ‘‘light’’ varieties of foods and beverages, which are similar to the original varieties but with fewer calories.
The prices of ‘‘light’’ products are very close to the prices of the original products.

The introduction of ‘‘light’’ varieties simply serves to expand the market.

While there is a need to capture such expansion when the weights are revised, the price changes for the existing items can be used to capture those of the ‘‘light’’ items.

8.41 The second measurement concern with respect to new products is the incorporation of the eﬀect of those products at launch.
The preceding discussion is concerned with the incorporation of price changes into the index once two successive quotations are available.

Yet there is a gain to the consumer when comparing the price in the ﬁrst of these periods with the price in the period that preceded the introduction of the product.

This latter price is a hypothetical price.


It is the price which would make the demand from the community for the product equal to zero; that is, it is the reservation price which, when inserted into the demand function, sets demand to zero.

If a demand system can be estimated, so too can the reservation price.

The virtual reservation price is compared with the actual price in the period of introduction, and this is used to estimate the surplus from the introduction of the good.
If the reservation price is relatively high, then the introduction of the new good is clearly of some beneﬁt to the consumer.

To ignore this beneﬁt, and the change from the shadow price to the actual price in its period of launch, is to ignore something of the price movements that give rise to improvements in the standard of living.
Of course, if a ‘‘new’’ good is a close substitute – at the price it is brought into the index – for goods already in existence, then no additional consumer surplus is generated.

8.42 It should be noted that a consumer may be in a geographical area in which a new good or service, say, cable television, a video rental outlet or health facility, is not present.
The beneﬁts of the new good on its intro- duction to diﬀerent geographical areas will therefore develop over time as the new good becomes more gen- erally available.

The beneﬁts will emerge again and again for each sector of the population that beneﬁts from access to the new product.

In practice, such items gain increasing weight as the index is rebased or the sample rotated.
8.43 The methods outlined below for the inclusion of substitute and new goods include both normal CPI procedures and exceptional treatments.

In regard to the former, consideration is given in paragraphs 8.44 to 8.58 to the rebasing of the index, rotating of items, intro- duction of new goods as replacements for discontinued ones on rotating, and a strategy for dealing with new item bias.

In regard to the latter, techniques that require diﬀerent sets of data are outlined.
The use of chained matched models and hedonic indices was discussed in ITEM SUBSTITUTION, SAMPLE SPACE AND NEW PRODUCTS Chapter 7 in the context of products experiencing rapid turnover in models.

Analytic frameworks that consider new goods bias by way of reservation prices and sub- stitution eﬀects are considered in paragraphs 8.59 and 8.60 and Appendix 8.2.

The data requirements and econometric expertise are much more demanding for these approaches.
Sample rebasing and rotation 8.44 A new good may be readily incorporated in the index at the time of rebasing the index, or when the whole or the pertinent part of the sample is rotated.

If the new good has, or is likely to have, substantial sales, and is not a replacement for a pre-existing one, or is likely to command a much higher or lower market share than the pre-existing one it is replacing, then new weights are necessary to reﬂect this.
New weights are only fully available on rebasing, not on sample rotation.

There will thus be a delay in the new item’s inclusion in the index.

The extent of the delay will depend on how close the introduction of the item is to the next rebasing and, more generally, the frequency with which the index is rebased.

This discussion of rebasing is eﬀectively concerned with the use of new weights for the index.
Even if the index is rebased annually and chained, there will be a delay until the annual rebasing before weights can be assigned, and there may even be a further six-month delay for the sampling and collating of the survey results for the weights.

Such frequent rebasing allows for the early introduction of a new good and is to be advised when the weights are not keeping pace with product innovations.
8.45 At the elementary level of aggregation, an implicit weight equal to the expenditure share is given by the Jevons index, for example, to each price relative.

The Dutot index gives each price change the weight of its price relative to the sum of the prices in the initial base period of the comparison (see Chapter 7).

If a product area is expected to be subject to dynamic innovations, then the sample may be increased on rotation, without any changes to the weight for the group.
There would simply be more items selected to form the arithmetic or geometric average price change.

As new varieties become available, they could be substituted for some of the existing ones, there being a wider range from which to draw a comparable item, or less eﬀort involved in the quality adjustment procedure for a non-comparable one.

8.46 Some statistical agencies rotate (resample) items within product groups.
Opportunities exist to introduce new items within a weighted group under such circum- stances.

The resource practicalities of such schemes require items to be rotated on a staggered basis for dif- ferent product groups.

Product groups experiencing rapid change should be rotated more frequently.

The incorporation of new goods using sample rotation allows some of the existing weight of the product group to be re- allocated to the new good.
Yet it implicitly uses the overlap method for the introduction of the new good of a diﬀerent quality.

The diﬀerence in prices in the overlap period of the new and obsolete items is assumed to be equal to their quality diﬀerence.

The assumptions implicit in such procedures have been outlined above and 143 CONSUMER PRICE INDEX MANUAL: THEORY AND PRACTICE their likely veracity needs to be considered.
Since evo- lutionary items are deﬁned as continuations of the ser- vice ﬂow of existing (and possibly exiting) ones, the hedonic framework may be more suitable in some cases to the use of the overlap method.

These and other methods and the choice between methods are discussed in Chapter 7.

8.47 In many countries, rebasing is infrequent and sample rotation is not undertaken, despite their advan- tages.

The rotating of samples on a frequent basis should not, however, be considered a panacea.
Sample rotation is an arduous task, especially when performed over a range of product groups experiencing rapid change.

Even frequent rotation, say, every four years, may miss many new goods.

Yet it is not necessary for statistical agencies to wait until an item is obsolete before a new one is introduced.

It is quite feasible for statistical agencies to pre-empt the obsolescence of an old item and decide on its early substitution by a new one.
In some product areas, the arrival of a new good is well advertised in advance of the launch.

In other areas, it is feasible for a statistical agency to have general procedures for sub- stitutions, as outlined below.

Without such a strategy, and where rotation or rebasing is infrequent, a country would be open to serious new product bias.
8.48 In summary: The treatment of a new good as a replacement for an existing one can be undertaken if the old item’s weight suitably reﬂects the new good’s sales and if a suitable quality adjustment can be made to its price to link it to the existing, old, price series.

 If the new good does not ﬁt into the pre-existing weighting structure, it can be included on rebasing, though this may be infrequent in some countries.
Regular sample rotation provides a means by which the inclusion of such items can be formally recon- sidered, though since this is undertaken on a staggered basis, only the weights within the product group are reallocated, not those between the groups.

 Directed sample substitution, as opposed to waiting for sample rotation, may be used to pre-empt the arrival of new goods.

 Revolutionary items will not ﬁt into existing weight- ing structures and alternative means are required.
The modiﬁed short-run or chained framework out- lined in Chapter 7, paragraphs 7.153 to 7.173, may be more appropriate for product areas with high turn- over of items.

Directed replacements for evolutionary items and di- rected augmentation of the sample for revolutionary goods are considered below.
Directed replacements and sample augmentation 8.49 For evolutionary goods in product areas where there is a rapid replacement and introduction of such goods, a policy of directed replacements might be adopted.

Judgement, experience, discussions with store managers, market research companies and a statistical metadata system should help identify such products.

The selection 144 of replacements is directed to evolutionary items in order to ensure that the index maintains its representativity.
If the new version of a product is designed as a replacement for an existing one, then substitution might be auto- matic.

Once a substitute has been made, the prices require an adjustment for the quality diﬀerence using, perhaps, the overlap method, imputation, or an explicit estimate based on production or option costs, or a hedonic regression as discussed in Chapter 7.

8.50 The management of the directed substitution can take a number of forms.
It can comprise instructions to price collectors who are informed of deﬁned conﬁg- urations of a product, such as ‘‘high end’’, ‘‘main- stream’’, ‘‘economy’’, ‘‘entry level’’ and ‘‘other’’ (Lane, 2001).

Directions might also be given as to the propor- tions expected of items at these levels, say, 20 per cent of the market should be ‘‘high end’’.

Such informa- tion should be based on actual data or judgement of specialists.

The conﬁgurations are revised, say, every six months.
What was ‘‘high end’’ at the start of the period may now be ‘‘entry level’’ and the price collectors will have new conﬁgurations indicating what the desired replacements should look like.

They are directed to particular replacements.

Alternatively, the price collector might be responsible for the selection of replacements, either after discussion with store managers or, if an indication of the market share of the popular makes is given, with probability proportionate to size.
There are, of course, other variants.

In such markets, the desired end eﬀect is that replacement items likely to be repre- sentative of substantial sales are selected and that this selection is made earlier rather than later.

The point is not to miss the birth of such items and to facilitate quality adjustment.
8.51 It is important to emphasize that, on the intro- duction of new versions of these evolutionary goods, a particularly high price may be charged to take advantage of segments of the market willing to pay a premium for the ‘‘newness’’ of the item.

Alternatively, a particularly low price may be charged to introduce the good to the market in order to help gain acceptance.
After a while, prices may be changed as the novelty of the item wears oﬀ or as it gains acceptance, or as competitors bring out improved products.

Directed substitution is important in ensuring that the CPI captures the unusual price increases at the launch.

It is also necessary in ensuring that the coverage of items becomes more representa- tive.

Although directed substitution allows for both, a caveat applies.
If the overlap method is used, the item is introduced on the assumption that the price diﬀerence between the old and new items equates to their quality diﬀerence.

For example, if a new type of detergent is introduced with a new, biological cleaning action, it may be that the typical consumer is willing to pay a price of 10 against the existing standard detergent’s price of 8.
With no explicit estimate of the additional usefulness or utility to be gained from the biological action, the overlap method implicitly assumes it is worth 2.

Yet there may be an introductory launch price of 8 and the price may later increase to 10.

At the time of the overlap, the two prices would be the same, there being no adjudged quality diﬀerence.

In fact, the quality-adjusted price would be falling; there is a quality diﬀerence of 2, but this cannot be deduced by the statistical oﬃce.
In general, therefore, when there is evidence of items being launched at unusual prices and the overlap method is used, it is better to make the replacement later when the market has settled.

8.52 For revolutionary goods, substitution may not be appropriate.

First, they may not be able to be deﬁned within the existing classiﬁcation systems.

Second, a major part of their uniqueness may be the manner in which they are sold, which will require extending the sample to such new sales channels.
Third, there will be no previous items to match such goods against in order to make a quality adjustment to prices since, by deﬁnition, they are sub- stantially diﬀerent from pre-existing goods.

Finally, there is no weight to attach to the new outlets or items.

8.53 The ﬁrst need is to identify new goods.

The suggested contacts with market research companies, outlet managers and manufacturers, mentioned above in relation to producing a supportive metadata system, are also pertinent here.
Once the new goods are identiﬁed, sample augmentation is appropriate for the introduction of revolutionary goods.

It is necessary to bring the new revolutionary goods into the sample, in addition to what already exists in the sample.

This may involve extending the classiﬁcation, the sample of outlets, and the item list within new or existing outlets.

The choice of means by which the new goods are introduced is more problematic.
8.54 Once two price quotes are available, it should be possible to splice the new good onto an existing or obsolete one.

This of course misses the impact of the new item in its initial period.

As discussed below, however, including such eﬀects is not a trivial exercise.

Consider the linking of a good that is likely to be replaced in the market by the new good.
For example, a relatively new electrical kitchen appliance might follow the price index for existing kitchen appliances up to the period of the link, and then the price changes for the new good in subsequent periods.

This would create a separate and additional price series for a new good that augments the sample, as illustrated in Table 8.1.

Item C is new in period 2 and has no base period weight.
Its price change between periods 1 and 2, had it existed, is assumed to follow the overall index for products A and B.

For period 3 onwards, a new linked price series is formed for C, which for period 3 is 101.40 0.985 = 99.88 and for period 4 is 101.40 0.98 = 99.37.

New revised weights in period 2 show C’s weight to be 20 per cent of all the items.
The new index for period 3 is: 101:40 [0:8 (101:9=101:4)+0:2 (99:88=101:4)] =0:8 101:9+0:2 99:88=101:50 and for period 4: 101:40 [0:8 (102:7=101:4)+0:2 (99:37=101:4)] =0:8 102:7+0:2 99:37=102:05 8.55 If C were an evolutionary good replacing B, then there would be no need to introduce new weights and no need to augment the sample.
The revolutionary good C has no weight in the base period; the splicing thus ITEM SUBSTITUTION, SAMPLE SPACE AND NEW PRODUCTS Table 8.1 Example of sample augmentation Products Base weight Revised weight Period 1 Period 2 Period 3 Period 4 0.6 0.4 A B All items C Spliced C All items (revised) 0.5 0.3 0.8 0.2 100.00 100.00 100.00 100.00 100.00 101.00 102.00 101.40 100.00 101.40 101.40 101.50 102.50 101.90 98.50 99.88 101.50 102.50 103.00 102.70 98.00 99.37 102.05 8.56 requires a revision of the weights at the same time.
Both the selection of the series onto which the new item is spliced, and the product groups selected for the weight revision, require some judgement.

Items whose market share is likely to be aﬀected by the introduction of the new good should be selected.

If the new good is likely to be responsible for a signiﬁcant share of expenditure, such that it will aﬀect the weights of a broad class of product groups, then there may be a case for a realignment of the overall weighting procedure.
Such seismic shifts can of course occur, especially in the communications indus- tries, and for a wider range of markets when trade bar- riers are relaxed in less-developed economies or when regulations are removed.

The change in weights may also be required for disappearing goods no longer sold in an economy.

In that case, the weights of these goods need to be reassigned.
As noted in Chapter 7, paragraphs 7.132 to 7.158, chaining and hedonic indices may well be appropriate when there is a rapid turnover in such new and obsolete goods.

Chaining is an extension of the above procedure and can be used to introduce a new good as soon as it is available for two successive periods.

Item augmentation may also be used for evo- lutionary goods that are likely to be responsible for a substantial share of the market, while not displacing the existing goods.
Say, for example, that a country has a local brewery and that a licensing agreement with a foreign brewery has led to the joint production of two beers, under diﬀerent brand names.

Say the market share for beer from the brewery remains the same, but one segment of the market now drinks foreign as opposed to domestic beer.

Price collectors may be directed to a forced substitution of some of the sample of domestic beers for foreign ones, the weight remaining the same.
This would be similar to a quality adjustment using a non-comparable replacement, as discussed in Chapter 7, paragraphs 7.72 to 7.115.

Alternatively, the sample may be augmented since there is concern that a smaller sam- ple of domestic beers may now not be suﬃciently representative.

The augmentation process may be similar to that outlined in Table 8.1, with the new foreign beer C accounting for 20 per cent of the market.
If the advent of foreign beers displaced some of the alcoholic spirits market, say, then the revision of weights would extend into that product group.

As noted in Chapter 7, para- graphs 7.125 to 7.158, chaining and hedonic indices may well be appropriate when there is a rapid turnover in such new and obsolete goods.

With chaining, the good needs to be available for only two successive periods to allow for its introduction.
145 CONSUMER PRICE INDEX MANUAL: THEORY AND PRACTICE 8.57 In some instances a directed replacement is required for evolutionary and revolutionary outlets.

Forced augmentation of the sample of outlets may be implemented so that new goods available only in speciﬁc outlets are included.

This is especially likely in the service sector, where a new service is particular to speciﬁc out- lets, for example cyber cafe´ s or online retailers.

The procedures are similar to those described for items.
For example, in the above example, instead of products A, B and C, consider C as a new outlet in addition to outlets A and B.

Some estimate would be required of its expected sales share to form the revised weights.

8.58 The eﬀect of a new outlet on the index depends on how it is included, as well as the nature of the market and its reaction to the new outlet.

First, if a new outlet oﬀers some innovation which induces some consumers to shop there, there is an increase in usefulness or uti- lity.
Because of imperfect knowledge about the new outlet or diﬀerent preferences of diﬀerent segments of the market, the old outlet may not close down.

There is no natural prompt for the new outlet to be introduced into the CPI, as there is with the closure of an old outlet.

The start-up of the new outlet may have been apparent to the statistical oﬃce.

If the new outlet is expected to have substantial sales, it may augment the sample.

It may be spliced onto the index in the manner of item C above.
Such a methodology would not include the welfare gain to consumers arising from the uniqueness of the outlet (Trajtenberg, 1989), since price comparisons are only being undertaken once it has been introduced.

The initial welfare eﬀect is between the period prior to its existence and the period of its introduction.

Second, all other outlets might lower their quality-adjusted prices to match those from the new outlet.
The fall in price and gain in usefulness or utility arising from the new outlet’s technology would then be captured by the CPI.

Finally, outlets may appear that oﬀer a wider range of options in terms of goods and service, which is valued by consumers and is therefore an improvement in the standard of living via the gain in utility.

There is nothing in current CPI methodology that allows for the valuation of such gains (Shapiro and Wilcox, 1997a).
Reservation prices 8.59 Shapiro and Wilcox (1997a, p. 144) expressed concerns over: .

.

.

the rare new item that delivers services radically diﬀerent from anything previously available.

For exam- ple, even the earliest generation of personal computers allowed consumers to undertake tasks that previously would have been prohibitively expensive.

This problem can be solved only by estimating the consumer surplus created by the introduction of each new item.
Hausman (1994) [republished as Hausman (1997)] argues that this must involve explicit modeling of the demand for each new item.

Although explicit modeling of demand may be of dubious practicality for widespread implementation in the CPI, strategic application in a few selected cases might be worthwhile.

8.60 The technical means for such estimates is recognized as being beyond the practical capabilities of a 146 statistical agency.
More disturbing is that the argument for the inclusion of such eﬀects extends from revolu- tionary new goods to the clutter of evolutionary items such as new breakfast cereals.

Appendix 8.2 provides some details of a generalized Laspeyres approach which takes account of substitution between new and old models.

Given the complexity of the estimation systems involved, however, this manual envisages a pragmatic approach which would initially exclude such eﬀects.
Summary 8.61 The need to consider the sample space of the items selected by the index number methodology and new goods arises out of a very real concern with the dynamic nature of modern markets.

New goods and quality changes are far from being a new phenomenon.

As Triplett (1999) has argued, it has not been demon- strated that the rate of new product development and introduction is much higher now than in the past.
It is certainly accepted, however, that the number of new products and varieties is substantially greater than before.

Computer technology provides cost-eﬀective means for collecting and analysing very large sets of data.

Chapter 6 considers the use of hand-held com- puters for data capture, and the availability of bar-code scanner data.
The proper handling of such data requires consideration of aspects beyond those normally taken into account in regard to the static intersection universe which underscores matched samples.

Appendix 8.1 to this chapter provides an outline of these sampling issues.

8.62 The following important points should be borne in mind: Where nothing much in the quality and range of goods available changes, use of the matched models meth- od presents many advantages.
The matched models method compares like with like, from like outlets.

 Statistical metadata systems are needed to help iden- tify the product areas in which matching provides few problems, and to focus attention on those areas that are problematic.

They show how to collect and provide the information that will facilitate quality adjustment.

They also allow for transparency in methods and they facilitate retraining.
Where there is a very rapid turnover in items such that serious sample depletion takes place quickly, replace- ments cannot be relied upon to make up the sample.

Alternative mechanisms, which sample from or use the double universe of items in each period, are required.

These include chained formulations and hedonic indices, as discussed in Chapter 7, paragraphs 7.125 to 7.158.
Some new goods can be treated as evolutionary and replacements incorporated using non-comparable with associated quality adjustments.

The timing of the replacement is critical for both the eﬃcacy of the quality adjustment and the representativity of the index.

 Instructions to price collectors on the selection of re- placements are important, for they too have a bearing ITEM SUBSTITUTION, SAMPLE SPACE AND NEW PRODUCTS on the representativity of the index.
The replacement of obsolete items with newly introduced items, in turn, leads to diﬃculties in undertaking quality adjustments, while their replacement with similar items leads to problems with representativity.

 Sample rotation is an extreme form of the use of replacements, and is one mechanism for refreshing the sample and thus increasing its representativity.
Against is the possibility of bias arising from the implicit assumptions underlying the overlap procedure for quality adjustment not being met.

this, however, Revolutionary goods may require the augmentation of the sample to make room for new price series and new weighting procedures.

The classiﬁcation of new goods into evolutionary goods and revolutionary goods has a bearing on the strategy for their introduction, directed replacement (substitution) and sample augmentation.
The initial gain in consumer welfare arising from new items and loss in welfare because items disappear are not captured by either of these procedures.

Econo- metric estimates of reservation prices provide an approach that is theoretically appropriate, although problematic in practice.

147 CONSUMER PRICE INDEX MANUAL: THEORY AND PRACTICE Appendix 8.1 Appearance or disappearance of products or outlets 1.
In previous chapters, it was generally assumed that the target quantity for estimation could be deﬁned in terms of a ﬁxed set of products.

Here we consider the complications arising from the fact that the products and outlets are con- tinually changing.

The rate of change is rapid in many indus- tries.

Sampling to estimate price changes is thus a dynamic rather than a static problem.

Somehow, the prices of new products and the prices in new outlets have to be compared to old ones.
Whatever methods and procedures are used in a price index to handle these dynamic changes, the eﬀects of these procedures will always amount to an explicit or implicit esti- mation approach for this dynamic universe.

The representation of change in a price index 2.


From a sample selection perspective, there are three ways of handling dynamic changes in an elementary aggregate universe (Dale´ n, 1998a), where varieties and outlets move in and out: – by resampling the whole elementary aggregate at certain points in time; – by a one-to-one replacement of one variety or outlet for another; – by adding and deleting single observation points (items in outlets) within an index link.

Sample rotation 3.
Sample rotation 3.

By resampling it is meant that the old sample is reconsidered as a whole so as to make it representative of the universe in a later period.

This does not necessarily mean that all or even most sampling units have to be changed, only that a fresh look is taken at the representativity of the whole sample and changes are undertaken, as appropriate.

The methods used for resampling could be any of those used for the initial sampling.
In the case of probability sampling, every unit belonging to the universe in the later period needs to have a non-zero probability, equal to its relative market share, of being included in the sample.

4.

Resampling (or sample rotation) is traditionally combined with the overlap method outlined in Chapter 7, paragraphs 7.45 to 7.52.

It is a similar procedure to that used when combining two links in chain indices.
The ﬁrst period for which the new sample is used is also the last period for which the old sample is used.

Thereby, price change estimation is always based on one sample only – the old sample up to the overlap period and the new sample from the overlap period onwards (see below).

Resampling is the only method that is fully able to maintain the representativity of the sample.

Resources permitting, resampling should be under- taken frequently.
The appropriate frequency, of course, depends on the rate of change in a particular product group.

It also relies on the assumption that the price diﬀerences between the old and new items are appropriate estimates of quality diﬀerences.

At its extreme, resampling amounts to drawing a new sample in each period and comparing the average price between the samples, instead of the usual procedure of averaging price changes for matched samples.
Although logical from the point of view of representativity, resampling in each period would aggravate the problem of quality adjustment by its implicit procedure of quality adjustment, and is thus not recommended.

Replacements 5.

A replacement can be deﬁned as an individual successor to a sampled product that has either disappeared completely 148 from the market or lost market share in either the market as a whole or a speciﬁc outlet.

Criteria for selecting replacements may diﬀer considerably.
First, there is the question of when to make the replacement.

The usual practice is to do it either when an item disappears completely or when its share of the sales is reduced signiﬁcantly.

Another possible, but less-used rule, would be to replace an item when another variety within the same group, or representative item deﬁnition, has become larger with regard to sales, even if the old variety is still sold in signiﬁcant quantities.

6.

The second question is how to select the replacement item.
If the rule for initial selection was ‘‘most sold’’ or with probability proportionate to (sales) size, then the replacement rule could follow the same selection rule.

Alternatively, the replacement could be that item which is ‘‘most like’’ the old one.

The advantage of the former rule is that it produces better representativity.

The advantage of the ‘‘most like’’ rule is that, at least superﬁcially, it might reduce the quality adjustment problem.

7.
7.

It is important to realize that, under current conditions, replacements cannot adequately represent new items that are coming onto the market.

This is because what triggers a replacement is not the appearance of something new, but the disappearance or reduced importance of something old.

For example, if the range of varieties in a certain group is increas- ing, sampling can only represent this increase directly from the set of new varieties, say by sample rotation.

Adding and deleting 8.


It is possible to add a new observation point into an elementary aggregate within an index link.

For example, if a new brand or model of a durable is introduced without replacing any particular old model, it is desirable to add it to the sample, starting from the time of its introduction.

In order to accommodate this new observation in the index system, a reference price needs to be imputed.
A practical way to do this is to use the ratio of the price of the new item in the month of its introduction to the average of all other items in the ele- mentary aggregate from the reference period to the month of introduction.

In this way, the eﬀect of the new item on the index for months up to the introduction month will be neutral.

9.

Similarly, an item that disappears could just be deleted from the sample without replacement.

Price change can then be computed over the remaining items.
If no further action is taken, this means that the price change for the deleted item, which was measured up to the month prior to deletion, will be disregarded from the month of deletion.

This may or may not be desirable, depending on the circumstances in the particular product group.

Formulating an operational target in a dynamic universe 10.
A rigorous approach to statistical estimation requires an index estimation strategy, including both the operational target of measurement and the sampling strategy (design and estimator) needed for estimating this target.
This strategy would have to consist of the following components: – a deﬁnition of the universe of transactions or observation points (usually a product variety in an outlet) in each of the two time periods between which we want to estimate price change; – a list of all variables deﬁned for these units.

These variables should include prices and quantities (number of units sold at each price), but also all relevant price-determining char- acteristics of the products (and possibly also of the outlets).
This forms the price basis; – the target algorithm (index formula) that combines the values of the deﬁned variables for the observation points in the deﬁned universe into a single value; – procedures used for initial sampling of items and outlets and t. This index may also be called the identical units index.

It is equivalent to starting out with the observation points existing in s and then dropping (deleting) missing or dis- appearing points.
An example of such an index is: ITEM SUBSTITUTION, SAMPLE SPACE AND NEW PRODUCTS from the deﬁned universe; – procedures within the time span for replacing, sample rotation, adding or deleting observations; – the estimation algorithm (index formula) applied to the sample with the purpose of minimizing the expected error of the sample estimate compared with the target algorithm.
In principle, the estimation needs to consider all the procedures undertaken in replacement and sample rotation situations, including procedures for quality adjustment.

11.

Because of its complexity, the rigorous strategy out- lined above is generally not used in practical index construc- tion, although the associated information (statistical metadata) system is discussed in paragraphs 8.23 to 8.31 above.

A few comments on such possible strategies are made below.

A two-level aggregation system 12.


A starting point for discussing an objective of estimating a price index from a sample drawn from a dynamic universe is a two-level structuring of the universe of items and outlets that are considered in the scope of a price index.

These levels are: the aggregate level: at this level there is a ﬁxed structure of item groups h ¼ 1, .

.

.

, H (or perhaps a ﬁxed cross-structure of item groups by regions and outlet types) within an index link.
In terms of updating the universe of items, new goods and services would be deﬁned as new groups at the aggre- gate level and moved into the index only in connection with a new index link; the elementary level: at this level the aim is to capture the properties of a changing universe in the index by comparing new and old items.

The micro-comparison from period s to period t must be deﬁned so that new products or outlets enter the market and old products or outlets disappear from the market.

13.
13.

The common starting point for the three alternative approaches at the elementary level presented here is a basket index from period s to period t at the aggregate level: Ist= P h P h QhP t h QhPs h =P hW s hI st h , where W s h= QhPs h QhPs h P h and I st h = Pt h Ps h : (A8.1) h), Paasche (Qh=Qt hQt The quantities, Qh, are for h=1 .

.

.
.

.

H item groups from any period, or functions of quantities from several periods, for example, a symmetric average of the base and current periods s and t. Special cases of such a basket index are the Laspeyres (Qh=Qs h), Edgeworth (Qh=Qs h ) and h]1=2) price indices outlined in Chapters 15 to Walsh (Qh=[Qs 17.

Alternative formulations for an elementary-level estimation strategy now enter in the deﬁnition of I st h .
As a further common starting point, the set of items or outlets belonging to h in period u (= s or t) is deﬁned as Ou h. The concept of an observation point is introduced, usually a tightly speciﬁed item in a speciﬁc outlet.

For each observation point j 2 Ou j and a quantity sold qu j .

There are now three possibilities for deﬁning the operational target.

h, there is a price pu h+Qt The intersection universe 14.
The elementary index is deﬁned over the intersection universe, that is, only over observation points existing in both s I st h = P h\Ot j 2 Os P h\Ot j 2 Os h h qjpt j qjps j (A8.2) The intersection universe decreases successively over time, as fewer matches are found for each long-run comparison between s and t, s and t+1, s and t+2 etc., until it eventually becomes empty.
An attraction of the intersection universe is that there are, by deﬁnition, no replacements involved, and thus, nor- mally, no quality adjustments.

If the identical units index is combined with a short index link, followed by resampling from the universe in a later period, sampling from the intersection universe is a perfectly reasonable strategy, as long as the assumption implicit in the overlap procedure, that the price diﬀerences at that point in time reﬂect the quality diﬀerences, is valid.
The double universe h and Pt 15.

The polar opposite approach to the intersection uni- verse is to consider Ps h as average prices deﬁned over two separately deﬁned universes in the two periods.

A double universe could then be considered as the operational target of measurement: one universe in period s and another in period t. This seems to be a natural way of deﬁning the target, since both time periods should be of equal status and all products existing in either of them should be taken into account.
The diﬃculty with this approach is that the two universes are rarely comparable in terms of quality.

Some kind of adjust- ment for average quality change would need to be brought into the index.

The natural deﬁnition of the average prices involved in this approach is based on unit values.
This would lead to the following deﬁnition of a quality-adjusted unit value index: I st h = PPt h PPs hgst h , where PPt h= jpt qt j P j 2 Ot h P j 2 Ot h and PPs h= qt j j ps qs j P j 2 Os h P j 2 Os h (A8:3Þ qs j where g st h is the average quality change in h (also interpretable as a quality index), which of course needs further deﬁnition.

For example, g st h could be thought of as a hedonic adjustment procedure, where characteristics are held constant.
Equation (A8.3) was discussed in Chapter 7, paragraphs 7.142–7.149, as part of Laspeyres, Paasche, Fisher and To¨ rnqvist indices (as opposed to unit value ones), in a form which includes explicit hedonic quality adjustments, g st h .

This operational target is attractive for products where the rate of turnover of varieties is very fast, but where average quality changes only slowly or where reliable estimates of quality changes can be made.
The commonly used representative-item method is not really com- patible with a double universe target.

It implicitly focuses on pre-selected primary sampling units that are used for both period s and t. The replacement universe 16.

Neither sampling from the intersection universe nor from the double universe bears a close resemblance to usual practices for constructing price indices.

The most common 149 j < cqs no longer representative.
Mathematically, this could be deﬁned as follows: observation point j should be replaced in the ﬁrst period in which qt j , where c is a suitably chosen constant between 0 and 1 (a modiﬁcation being required for seasonal items).

The choice of replacement point would then be governed by a rule such as: aj should be chosen so that d( j, aj) is minimized for j.
Since some priority should be given to observation points that are important in terms of that deﬁnition can be modiﬁed to quantities or values, become, for example: aj should be chosen so that d(j; aj)=qt aj is minimized for j.

Other rules for the choice of replacement point or function to be minimized can of course be chosen.

19.
19.

The dissimilarity function needs to be speciﬁed; it may depend on the item group h. In general, it must be some kind of metric deﬁned on the set of characteristics of the product and outlet in question.

For example, priority could be given to dissimilarity either to ‘‘same outlet’’ or ‘‘same product’’, con- cepts which could easily be worked into such a metric.
A more troublesome concern is the inclusion of as many new points in Ot h as possible in the index deﬁnition, in order to ensure that the sample is representative.

As the above deﬁnitions now stand, the same new point could replace many predecessors, whereas there might be many new points which will not be sampled unless there is a need for a replacement.

This short- coming of the replacement universe is an inherent trait in the replacement method as such.
The replacement method is designed to maintain only the representativity of the old sample, not that of the new sample.

CONSUMER PRICE INDEX MANUAL: THEORY AND PRACTICE sampling method used in practice – the representative-item method combined with one-to-one replacements – needs a rationalization in terms of operational targets which diﬀers from these alternatives.

Such a rationalization of sampling from a replacement universe is considered below.

h and j 2= Ot 17.
h and j 2= Ot 17.

For each j 2 Os h and j 2 Ot h we deﬁne replacement items aj 2 Ot h whose price replaces that of j in the formula.

Obviously, for j 2 Os h, aj=j.

In addition to a replacement, a quality change from j to aj is included.

This gives rise to a quality adjustment factor gj, interpreted as the factor with which p s j must be multiplied for the consumer to be indiﬀerent between consuming items j and aj at prices ps j and pt aj.

I st h = qjpt aj qjps j gj P j 2 Os h P j 2 Os h : (A8.4) 18.
This step towards an operational use of the formula requires, ﬁrst, a deﬁnition of gj, which is possible using a hedonic regression as described in Chapter 7, paragraphs 7.132 to 7.152.

Second, there is a need to deﬁne aj.

A natural procedure is to use a dissimilarity function from j to aj.

The notation d( j, aj) is introduced for this function.

The common procedure of choosing the most similar item in cases of replacement now corresponds to minimizing the dissimilarity function.
Some further speciﬁcations nevertheless need to be made.

When is the replacement deﬁned to take place?

In practice, this ought to be done when the ﬁrst chosen variety is 150 Appendix 8.2 New goods and substitution 1.

An alternative approach to estimating the eﬀect of introducing new goods is to see new goods as a special case of substitution.

In each period a consumer, faced with a set of prices, decides what to consume.

The relative sales of the dif- ferent items sold may change over time.
Consumers may decide to consume less of one existing item and more of another existing one, or substitute consumption of an existing old item by a new one not previously available, or discontinue con- sumption of an existing item and substitute it by consumption of an existing or new one.

Such changes are generally prompted by changes in relative prices.
In many cases the ‘‘decision’’ of the consumer is tied to that of the producer or retailer, as items are no longer produced or sold so as to make way for new ones.

Such substitutions between items apply as much to radically new goods as to new models of existing goods.

In economic theory, the elasticity of substitution, denoted as s, is a measure of the change in the quantity of, say, item i relative to item j, that would arise from a unit change in the price of item i relative to item j.
A value of zero would imply that a change in price would lead to no substitution between the consumption of items and s>1 implies that the change in expenditure arising as a result of substituting items is positive: it is worth switching.

2.

There is an intuition here that, if s is known, and the extent to which substitutions occur in terms of their expendi- ture shares is also known, then estimates of the underlying price change that prompted the substitution can be derived.
This applies as much to substitution between existing items as to substitution between existing, discontinued and new ones.
The framework for operationalizing this institution for CPI use is proposed by Shapiro and Wilcox (1997b) – see also Lloyd (1975) and Moulton (1996a) – whereby the usual Las- peyres formulation is generalized to include the (demand) elasticity of substitution: " P n 2 0,t w0 1s pit pi0 #1=ð1sÞ (A8.5) where w0 are expenditure shares in the base period and the summation is over matched items available in both periods.
The correction, using s, incorporates a substitution eﬀect into the basic Laspeyres formula.

If s = 0, the formula is the traditional Laspeyres one.

As s !

1, the formula tends towards a base- period weighted geometric mean.

To use this formulation to generalize across the items in the summation, the restriction must apply that for any pair of items, the elasticity of sub- stitution must be the same.

The elasticity of substitution must also be the same over time.
Such forms are referred to as con- stant elasticity of substitution (CES) functional relationships.

3.

Feenstra (1994), Feenstra and Shiells (1997) and Balk (2000b) have extended the substitution to discontinued and new items.

The advantage of equation (A8.5) is that, given an esti- mate of s, a cost of living index which includes an estimate of substitution eﬀects can be measured in real time.

The incor- poration of the eﬀects of new and discontinued items follows directly from this.
Alternative frameworks for including sub- stitution eﬀects (discussed in Chapter 17) require expenditure data for the base and current periods.

4.

To extend the framework to new items, it is necessary to know how expenditures shift between new, existing and dis- ITEM SUBSTITUTION, SAMPLE SPACE AND NEW PRODUCTS continued items.
Let lt be the expenditure share of matched existing items out of the total in period t. The total includes existing and new items, so 1 lt is the share of new items in period t. Similarly, 1 l0 is the expenditure share of old, dis- continued items in period 0.
The generalized Laspeyres index, which includes substitution between existing and old and new items, is given by: " 1=ðs1Þ lt l0 P n 2 0,t w0 1s pit pi0 #1=ð1sÞ (A8.6) Like the usual Laspeyres index, it requires only the price relatives, the base period weights, the ratio of expenditure shares and an estimate of the elasticity of substitution.
It can be derived in a number of alternative forms, including gen- eralized, Paasche, Fisher or Sato–Vartia indices.

5.

While there is an intuition behind the above formula, its formal correspondence to an index of consumer prices deﬁned in economic theory is given by Balk (2000b).

De Haan (2001) shows how the Fisher equivalent could be derived from a decomposition of a Fisher index when there are new and dis- appearing goods.
The derivations show how the framework requires that s>1, a factor prompting Balk (2000b) to argue for its use for lower-level index aggregation, where this is more likely.

The remaining problems are the estimation of s, the availability of data on current expenditure shares, and the validity of the implied constant s. There are also some con- ceptual issues.

Increases in utility are regarded as having resulted from increases in the desirability of the items included in the above aggregation.
If such items improve, then utility increases.

Yet there are other goods outside the aggregation or system of demand equations.

Deterioration in such goods will lead to increases in the desirability of the included items and decreases in utility.
For example, if a consumer switches to private transport as a result of a deterioration in public trans- port, this should not be measured as a welfare gain resulting from better private transport, even though the expenditure ﬂows in equation (A8.6) shift that way (Nevo, 2001).

6.

The direct estimation of s requires considerable econo- metric expertise.

This puts it outside the routine construction of index numbers (see Hausman, 1997).

Balk (2000b) shows how an alternative numerical routine might work.
De Haan (2001) used scanner data to apply the methodology to a gen- eralized Fisher index.

He applied Balk’s routine to nine pro- duct groups, using data from the Netherlands CPI, and found values of s that exceeded unity.

He advised the use of chained indices to maximize the matching of ongoing items, a principle discussed in Chapter 7, paragraphs 7.153 to 7.158.
De Haan (2001) found major discrepancies between a generalized and ordinary Fisher index for at least six of the products, arguing for the need to incorporate the eﬀects of new goods (see also Opperdoes, 2001).

He also demonstrates how sensitive the procedure is to the selection of s: for a share in current expenditure for new items of 4.8 per cent, and s=1.2, a Paasche-type index which includes new goods would be 93 per cent below the Paasche price change for ongoing goods only.
For s=5.0 and the same expenditure share, the discrepancy falls to 34.1 per cent.

For very large values, say s>100, the two indices would be relatively close.

In such cases, the goods are almost identical, being near-perfectly substitutable; a switch to a new good would have little eﬀect, the new and existing goods having similar prices.
151 CALCULATING CONSUMER PRICE INDICES IN PRACTICE 9 Introduction 9.1 The purpose of this chapter is to provide a general description of the ways in which consumer price indices (CPIs) are calculated in practice.

The methods used in diﬀerent countries are not exactly the same, but they have much in common.

There is clearly interest from both compilers and users of CPIs in knowing how most sta- tistical oﬃces actually set about calculating their CPIs.
9.2 As a result of the greater insights into the prop- erties and behaviour of price indices that have been achieved in recent years, it is now recognized that some traditional methods may not necessarily be optimal from a conceptual and theoretical viewpoint.

Concerns have also been voiced in a number of countries about possible biases that may be aﬀecting CPIs.

These issues and concerns need to be considered in this manual.
Of course, the methods used to compile CPIs are inevitably con- strained by the resources available, not merely for col- lecting and processing prices, but also for gathering the expenditure data needed for weighting purposes.

In some countries, the methods used may be severely constrained by lack of resources.

9.3 The calculation of CPIs usually proceeds in two stages.

First, price indices are estimated for the elemen- tary expenditure aggregates, or simply elementary aggre- gates.
Then these elementary price indices are averaged to obtain higher-level indices using the relative values of the elementary expenditure aggregates as weights.

This chapter starts by explaining how the elementary aggre- gates are constructed, and what economic and statistical criteria need to be taken into consideration in deﬁning the aggregates.
The index number formulae most com- monly used to calculate the elementary indices are then presented, and their properties and behaviour illustrated using numerical examples.

The pros and cons of the various formulae are considered, together with some alternative formulae that might be used instead.

The problems created by disappearing and new items are also explained, as well as the diﬀerent ways of imputing values for missing prices.
9.4 The second part of the chapter is concerned with the calculation of higher-level indices.

The focus is on the ongoing production of a monthly price index in which the elementary price indices are averaged, or aggregated, to obtain higher-level indices.

Price-updating of weights, chain linking and reweighting are discussed, with ex- amples being provided.

The problems associated with introduction of new elementary price indices and new higher-level indices into the CPI are also dealt with.
It is explained how it is possible to decompose the change in the overall index into its component parts.

Finally, the possibility of using some alternative and rather more complex index formulae is considered.

9.5 The chapter concludes with a section on data editing procedures, as these are an integral part of the process of compiling CPIs.

It is essential to ensure that the right data are entered into the various formulae.
There may be errors resulting from the inclusion of incorrect data or from entering correct data inappro- priately, and errors resulting from the exclusion of cor- rect data that are mistakenly believed to be wrong.

The section examines data editing procedures which try to minimize both types of errors.

The calculation of price indices for elementary aggregates 9.6 CPIs are typically calculated in two steps.

In the ﬁrst step, the elementary price indices for the elementary aggregates are calculated.
In the second step, higher- level indices are calculated by averaging the elementary price indices.

The elementary aggregates and their price indices are the basic building blocks of the CPI.

Construction of elementary aggregates 9.7 Elementary aggregates are groups of relatively homogeneous goods and services.

They may cover the whole country or separate regions within the country.

Likewise, elementary aggregates may be distinguished for diﬀerent types of outlets.
The nature of the elemen- tary aggregates depends on circumstances and the avail- ability of information.

Elementary aggregates may therefore be deﬁned diﬀerently in diﬀerent countries.

Some key points, however, should be noted: Elementary aggregates should consist of groups of goods or services that are as similar as possible, and preferably fairly homogeneous.

 They should also consist of items that may be expected to have similar price movements.
The objective should be to try to minimize the dispersion of price movements within the aggregate.

 The elementary aggregates should be appropriate to serve as strata for sampling purposes in the light of the sampling regime planned for the data collection.

9.8 Each elementary aggregate, whether relating to the whole country or an individual region or group of outlets, will typically contain a very large number of individual goods or services, or items.
In practice, only a small number can be selected for pricing.

When selecting the items, the following considerations need to be taken 153 CONSUMER PRICE INDEX MANUAL: THEORY AND PRACTICE into account: The items selected should be ones for which price movements are believed to be representative of all the products within the elementary aggregate.
The number of items within each elementary aggregate for which prices are collected should be large enough for the estimated price index to be statistically reliable.

The minimum number required will vary between elementary aggregates depending on the nature of the products and their price behaviour.

 The object is to try to track the price of the same item over time for as long as possible, or as long as the item continues to be representative.
The items selected should therefore be ones that are expected to remain on the market for some time, so that like can be compared with like.

9.9 The aggregation structure.

The aggregation structure for a CPI is illustrated in Figure 9.1.
Using a classiﬁcation of consumers’ expenditures such as the Classiﬁcation of Individual Consumption according to Purpose (COICOP), the entire set of consumption goods and services covered by the overall CPI can be divided into groups, such as ‘‘food and non-alcoholic beverages’’.

Each group is further divided into classes, such as ‘‘food’’.

For CPI purposes, each class can then be further divided into more homogeneous sub-classes, such as ‘‘rice’’.
The sub-classes are the equivalent of the basic headings used in the International Comparison Program (ICP), which calculates purchasing power parities (PPPs) between countries.

Finally, the sub-class may be further sub- divided to obtain the elementary aggregates, by dividing according to region or type of outlet, as in Figure 9.1.

In some cases, a particular sub-class cannot be, or does not need to be, further subdivided, in which case the sub-class becomes the elementary aggregate.
Within each elemen- tary aggregate, one or more items are selected to represent all the items in the elementary aggregate.

For example, the elementary aggregate consisting of rice sold in super- markets in the northern region covers all types of rice, from which parboiled white rice and brown rice with over 50 per cent broken grains are selected as representative items.

Of course, more representative items might be selected in practice.
Finally, for each representative item, a number of speciﬁc products can be selected for price collection, such as particular brands of parboiled rice.

Again, the number of sampled products selected may vary depending on the nature of the representative product.

9.10 Methods used to calculate the elementary indi- ces from the individual price observations are discussed below.
Working upwards from the elementary price indices, all indices above the elementary aggregate level are higher-level indices that can be calculated from the elementary price indices using the elementary expendi- ture aggregates as weights.

The aggregation structure is consistent, so that the weight at each level above the elementary aggregate is always equal to the sum of its components.
The price index at each higher level of aggregation can be calculated on the basis of the weights and price indices for its components, that is, the lower- level or elementary indices.

The individual elementary price indices are not necessarily suﬃciently reliable to be 154 published separately, but they remain the basic building blocks of all higher-level indices.

9.11 Weights within elementary aggregates.


In most cases, the price indices for elementary aggregates are calculated without the use of explicit expenditure weights.

Whenever possible, however, weights should be used that reﬂect the relative importance of the sampled items, even if the weights are only approximate.

Often, the elemen- tary aggregate is simply the lowest level at which reliable weighting information is available.
In this case, the ele- mentary index has to be calculated as an unweighted average of the prices of which it consists.

Even in this case, however, it should be noted that when the items are selected with probabilities proportional to the size of some relevant variable such as sales, weights are impli- citly introduced by the sampling selection procedure.
9.12 For certain elementary aggregates, information about sales of particular items, market shares and re- gional weights may be used as explicit weights within an elementary aggregate.

Weights within elementary aggre- gates may be updated independently and possibly more often than the elementary aggregates themselves (which serve as weights for the higher-level indices).

9.13 For example, assume that the number of sup- pliers of a certain product such as fuel for cars is limited.
The market shares of the suppliers may be known from business survey statistics and can be used as weights in the calculation of an elementary aggregate price index for car fuel.

Alternatively, prices for water may be col- lected from a number of local water supply services where the population in each local region is known.
The relative size of the population in each region may then be used as a proxy for the relative consumption expendi- tures to weight the price in each region to obtain the elementary aggregate price index for water.

9.14 A special situation occurs in the case of tariﬀ prices.

A tariﬀ is a list of prices for the purchase of a particular kind of good or service under diﬀerent terms and conditions.

One example is electricity, where one price is charged during daytime while a lower price is charged at night.
Similarly, a telephone company may charge a lower price for a call at the weekend than in the rest of the week.

Another example may be bus tickets sold at one price to ordinary passengers and at lower prices to children or old age pensioners.

In such cases, it is appropriate to assign weights to the diﬀerent tariﬀs or prices in order to calcu- late the price index for the elementary aggregate.
9.15 The increasing use of electronic points of sale in many countries, in which both prices and quantities are scanned as the purchases are made, means that valuable new sources of information may become increasingly available to statistical oﬃces.

This could lead to sig- niﬁcant changes in the ways in which price data are collected and processed for CPI purposes.

The treatment of scanner data is examined in Chapters 7, 8 and 21.
Construction of elementary price indices 9.16 An elementary price index is the price index for an elementary aggregate.

Various diﬀerent methods and formulae may be used to calculate elementary price indices.
The methods that have been most commonly CALCULATING CONSUMER PRICE INDICES IN PRACTICE Figure 9.1 Typical aggregation structure of a consumer price index (CPI) OVERALL CPI products GROUP Food and non- alcoholic beverages GROUP Alcoholic beverages and tobacco OTHER GROUPS CLASS Bread and cereals CLASS Meat OTHER CLASSES SUB-CLASS Rice SUB-CLASS Bread OTHER SUB- CLASSES Sold in northern region Sold in southern region Sold in other regions ELEMENTARY AGGREGATE Rice sold in northern supermarkets ELEMENTARY AGGREGATE Rice sold in other northern outlets REPRESENTATIVE PRODUCT Parboiled long-grain white rice REPRESENTATIVE PRODUCT Brown rice: over 50 per cent broken rice SAMPLED PRODUCT Brand A SAMPLED PRODUCT Brand B used are illustrated by means of a numerical example in Table 9.1.
In the example, it is assumed that prices are collected for four items within an elementary aggregate.

The quality of each item remains unchanged over time, so the month-to-month changes compare like with like.

It is assumed initially that prices are collected for all four items in every month covered, so that there is a complete set of prices.

There are no disappearing items, no missing prices and no replacement items.
This is quite a strong assumption because many of the problems encountered in practice are attributable to breaks in the continuity of the price series for the individual items for one reason or another.

The treatment of disappearing and replacement items is taken up later.

It is also assumed that there are no explicit weights available.

9.17 Three widely used formulae that have been, or still are, in use by statistical oﬃces to calculate elemen- tary price indices are illustrated in Table 9.1.
It should be noted, however, that these are not the only possibili- ties and some alternative formulae are considered later.

 The ﬁrst is the Carli index for i=1,...., n items.
It is deﬁned as the simple, or unweighted, arithmetic mean 155 CONSUMER PRICE INDEX MANUAL: THEORY AND PRACTICE Table 9.1 Calculation of price indices for an elementary aggregate1 January February March April May June July 6.60 7.70 2.20 5.50 5.50 4.98 1.10 1.07 0.73 1.10 1.10 1.10 1.10 1.10 100.07 129.02 110.00 103.77 110.00 110.00 98.70 110.00 110.00 Item A Item B Item C Item D Arithmetic mean prices Geometric mean prices Item A Item B Item C Item D 6.00 7.00 2.00 5.00 5.00 4.53 1.00 1.00 1.00 1.00 6.00 7.00 3.00 5.00 5.25 5.01 7.00 6.00 4.00 5.00 5.50 5.38 Prices 6.00 7.00 5.00 4.00 5.50 5.38 6.00 7.00 2.00 5.00 5.00 4.53 6.00 7.20 3.00 5.00 5.30 5.05 Month-to-month price ratios 1.17 0.86 1.33 1.00 1.00 1.00 0.40 1.25 0.86 1.17 1.25 0.80 1.00 1.00 1.50 1.00 Current-to-reference-month (January) price ratios 1.00 1.00 1.50 1.00 1.00 1.00 2.50 0.80 1.00 1.00 1.00 1.00 1.17 0.86 2.00 1.00 1.00 1.03 1.50 1.00 1.00 1.03 1.50 1.00 1.00 1.00 1.00 1.00 Item A Item B Item C Item D Carli index – the arithmetic mean of price ratios Month-to-month index Chained month-to-month index Direct index on January Dutot index – the ratio of arithmetic mean prices Month-to-month index Chained month-to-month index Direct index on January Jevons index – the ratio of geometric mean prices = geometric mean of price ratios Month-to-month index Chained month-to-month index Direct index on January 112.50 112.50 112.50 100.00 100.00 100.00 110.67 110.67 110.67 105.00 105.00 105.00 100.00 100.00 100.00 100.00 100.00 100.00 108.93 122.54
All price indices have been calculated using unrounded ﬁgures.
of the price relatives, or price ratios, for the two periods, 0 and t, to be compared: I 0:t C = 1 n P pt i p0 i (9.1) The second is the Dutot index, deﬁned as the ratio of the unweighted arithmetic mean prices: I 0:t D = P pt i P p0 i 1 n 1 n (9.2) The third is the Jevons the unweighted geometric mean of the price relative or ratio which is identical to the ratio of the unweighted geometric mean prices: index, deﬁned as 1=n J =Q pt I 0:t i p0 i = Q ( pt Q ( p0 i)1=n i )1=n (9.3) 9.18 The properties of the three indices are examined and explained in some detail in Chapter 20.
Here, the purpose is to illustrate how they perform in practice, to compare the results obtained by using the diﬀerent for- mulae and to summarize their strengths and weaknesses.

9.19 Each month-to-month index shows the change in the index from one month to the next.

The chained monthly indices link together these month-to-month changes by successive multiplication.

The direct indices compare the prices in each successive month directly with those of the reference month, January.
By simple inspection of the various indices, it is clear that the 156 choice of formula and method can make a substantial diﬀerence to the results obtained.

Some results are striking, in particular the large diﬀerence between the chained Carli index for July and each of the direct indices for July, including the direct Carli.

9.20 The properties and behaviour of the diﬀerent indices are summarized in the following paragraphs (see also Chapter 20).
First, the diﬀerences between the results obtained by using the diﬀerent formulae tend to increase as the variance of the price relatives, or ratios, increases.

The greater the dispersion of the price move- ments, the more critical the choice of index formula, and method, becomes.

If the elementary aggregates are deﬁned in such a way that the price movements within the aggregate are minimized, the results obtained become less sensitive to the choice of formula and method.
9.21 Certain features displayed by the data in Table 9.1 are systematic and predictable; they follow from the mathematical properties of the indices.

For example, it is well known that an arithmetic mean is always greater than, or equal to, the corresponding geometric mean, the equality holding only in the trivial case in which the numbers being averaged are all the same.
The direct Carli indices are therefore all greater than the Jevons indices, except in May and July when the four price relatives based on January are all equal.

In general, the Dutot may be greater or less than the Jevons, but tends to be less than the Carli.

9.22 One general property of geometric means should be noted when using the Jevons index.

If any one observation out of a set of observations is zero, their geometric mean is zero, whatever the values of the other observations.
The Jevons index is sensitive to extreme falls in prices and it may be necessary to impose upper and lower bounds on the individual price ratios of say 10 and 0.1, respectively, when using the Jevons.

Of course, extreme observations often result from errors of one kind or another, so extreme price movements should be carefully checked anyway.

9.23 Another important property of the indices illustrated in Table 9.1 is that the Dutot and the Jevons indices are transitive, whereas the Carli is not.
Transi- tivity means that the chained monthly indices are iden- tical to the corresponding direct indices.

This property is important in practice, because many elementary price indices are in fact calculated as chain indices which link together the month-on-month indices.
The intransitivity of the Carli index is illustrated dramatically in Table 9.1 when each of the four individual prices in May returns to the same level as it was in January, but the chain Carli registers an increase of almost 14 per cent over January.

in July, although each individual price is Similarly, exactly 10 per cent higher than in January, the chain Carli registers an increase of 29 per cent.
These results would be regarded as perverse and unacceptable in the case of a direct index, but even in the case of a chain index the results seems so intuitively unreasonable as to undermine the credibility of the chain Carli.

The price changes between March and April illustrate the eﬀects of ‘‘price bouncing’’ in which the same four prices are observed in both periods but they are switched between the diﬀerent items.
The monthly Carli index from March to April increases whereas both the Dutot and the Jevons indices are unchanged.

9.24 The message emerging from this brief illustra- tion of the behaviour of just three possible formulae is that diﬀerent index numbers and methods can deliver very diﬀerent results.
Index compilers have to familiarize themselves with the interrelationships between the vari- ous formulae at their disposal for the calculation of the elementary price indices so that they are aware of the implications of choosing one formula rather than another.

Knowledge of these interrelationships is never- theless not suﬃcient to determine which formula should be used, even though it makes it possible to make a more informed and reasoned choice.
It is necessary to appeal to other criteria in order to settle the choice of formula.

There are two main approaches that may be used, the axiomatic and the economic approaches.

Axiomatic approach to elementary price indices 9.25 As explained in Chapters 16 and 20, one way in which to decide upon an appropriate index formula is to require it to satisfy certain speciﬁed axioms or tests.
The tests throw light on the properties possessed by diﬀerent kinds of indices, some of which may not be intuitively obvious.
Four basic tests will be cited here to illustrate the axiomatic approach: Proportionality test – if all prices are l times the prices in the price reference period (January in the example), the index should equal l. The data for July, when every price is 10 per cent higher than in January, show CALCULATING CONSUMER PRICE INDICES IN PRACTICE that all three direct indices satisfy this test.
A special case of this test is the identity test, which requires that if the price of every item is the same as in the reference period, the index should be equal to unity (as in May in the example).

 Changes in the units of measurement test (commen- surability test) – the price index should not change if the quantity units in which the products are measured are changed (for example, if the prices are expressed per litre rather than per pint).
The Dutot index fails this test, as explained below, but the Carli and Jevons indices satisfy the test.

 Time reversal test – if all the data for the two periods are interchanged, then the resulting price index should equal the reciprocal of the original price index.

The Carli index fails this test, but the Dutot and the Jevons indices both satisfy the test.
The failure of the Carli to satisfy the test is not immediately obvious from the example, but can easily be veriﬁed by interchanging the prices in January and April, for example, in which case the backwards Carli for January based on April is equal to 91.3 whereas the reciprocal of the forwards Carli is 1/132.5 or 75.5.

 Transitivity test – the chain index between two periods should equal the direct index between the same two periods.
It can be seen from the example that the Jevons and the Dutot indices both satisfy this test, index does not.

For example, whereas the Carli although the prices in May have returned to the same levels as in January, the chain Carli registers 113.9.

This illustrates the fact that the Carli may have a sig- niﬁcant built-in upward bias.
9.26 Many other axioms or tests can be devised, but the above are suﬃcient to illustrate the approach and also to throw light on some important features of the elementary indices under consideration here.

9.27 The sets of products covered by elementary aggregates are meant to be as homogeneous as possible.

If they are not fairly homogeneous, the failure of the Dutot index to satisfy the units of measurement or commensurablity test can be a serious disadvantage.
Although deﬁned as the ratio of the unweighted arith- metic average prices, the Dutot index may also be interpreted as a weighted arithmetic average of the price ratios in which each ratio is weighted by its price in the base period.

This can be seen by rewriting formula (9.2) above as 1 n I 0:t D = i=p0 i ) P p0 1 n i ( pt P p0 i However, if the products are not homogeneous, the relative prices of the diﬀerent items may depend quite arbitrarily on the quantity units in which they are mea- sured.
9.28 Consider, for example, salt and pepper, which are found within the same sub-class of COICOP.

Sup- pose the unit of measurement for pepper is changed from grams to ounces, while leaving the units in which salt is measured (say kilos) unchanged.

As an ounce of pepper is equal to 28.35 grams, the ‘‘price’’ of pepper 157 CONSUMER PRICE INDEX MANUAL: THEORY AND PRACTICE increases by over 28 times, which eﬀectively increases the weight given to pepper in the Dutot index by over 28 times.
The price of pepper relative to salt is inherently arbitrary, depending entirely on the choice of units in which to measure the two goods.

In general, when there are diﬀerent kinds of products within the elementary aggregate, the Dutot index is unacceptable conceptually.

9.29 The Dutot index is acceptable only when the set of items covered is homogeneous, or at least nearly homogeneous.
For example, it may be acceptable for a set of apple prices even though the apples may be of diﬀerent varieties, but not for the prices of a number of diﬀerent kinds of fruits, such as apples, pineapples and bananas, some of which may be much more expen- sive per item or per kilo than others.

Even when the items are fairly homogeneous and measured in the same units, the Dutot’s implicit weights may still not be satisfactory.
More weight is given to the price changes for the more expensive items, but in practice they may well account for only small shares of the total expendi- ture within the aggregate.

Consumers are unlikely to buy items at high prices if the same items are available at lower prices.

9.30 It may be concluded that from an axiomaic viewpoint, both the Carli and the Dutot indices, although they have been, and still are, widely used by statistical oﬃces, have serious disadvantages.
The Carli index fails the time reversal and transitivity tests.

In principle, it should not matter whether we choose to measure price changes forwards or backwards in time.

We would expect the same answer, but this is not the case for the Carli.

Chained Carli indices may be subject to a sig- niﬁcant upward bias.

The Dutot index is meaningful for a set of homogeneous items but becomes increasingly arbitrary as the set of products becomes more diverse.
On the other hand, the Jevons index satisﬁes all the tests listed above and also emerges as the preferred index when the set of tests is enlarged, as shown in Chapter 20.

From an axiomatic point of view, the Jevons index is clearly the index with the best properties, even though it may not have been used much until recently.

There seems to be an increasing tendency for statistical oﬃces to switch from using Carli or Dutot indices to the Jevons index.
Economic approach to elementary price indices 9.31 In the economic approach, the objective is to estimate an economic index – that is, a cost of living index for the elementary aggregate (see Chapter 20).

The items for which prices are collected are treated as if they con- stituted a basket of goods and services purchased by consumers, from which the consumers derive utility.
A cost of living index measures the minimum amount by which consumers would have to change their expendi- tures in order to keep their utility level unchanged, allowing consumers to make substitutions between the items in response to changes in the relative prices of items.

In the absence of information about quantities or expenditures within an elementary aggregate, the index can only be estimated when certain special conditions are assumed to prevail.
158 9.32 There are two special cases of some interest.

The ﬁrst case is when consumers continue to consume the same relative quantities whatever the relative prices.

Consumers prefer not to make any substitutions in reponse to changes in relative prices.

The cross- elasticities of demand are zero.

The underlying pre- ferences are described in the economics literature as ‘‘Leontief ’’.

With these preferences, a Laspeyres index would provide an exact measure of the cost of living index.
In this ﬁrst case, the Carli index calculated for a random sample would provide an estimate of the cost of living index provided that the items are selected with probabilities proportional to the population expenditure shares.

It might appear that if the items were selected with probabilities proportional to the population quan- tity shares, the sample Dutot would provide an estimate of the population Laspeyres.
If the basket for the Las- peyres index contains diﬀerent kinds of products whose quantities are not additive, however, the quantity shares, and hence the probabilities, are undeﬁned.

9.33 The second case occurs when consumers are assumed to vary the quantities they consume in inverse proportion to the changes in relative prices.

The cross- elasticities of demand between the diﬀerent items are all unity, the expenditure shares being the same in both periods.
The underlying preferences are described as ‘‘Cobb–Douglas’’.

With these preferences, the geometric Laspeyres would provide an exact measure of the cost of living index.

The geometric Laspeyres is a weighted geometric average of the price relatives, using the expen- diture shares in the earlier period as weights (the expen- diture shares in the second period would be the same in the particular case under consideration).
In this second case, the Jevons index calculated for a random sample would provide an unbiased estimate of the cost of living index, provided that the items are selected with prob- abilities proportional to the population expenditure shares.
9.34 On the basis of the economic approach, the choice between the sample Jevons and the sample Carli rests on which is likely to approximate the more closely to the underlying cost of living index: in other words, on whether the (unknown) cross-elasticities are likely to be closer to unity or zero, on average.

In practice, the cross- elasticities could take on any value ranging up to plus inﬁnity for an elementary aggregate consisting of a set of strictly homogeneous items, i.e., perfect substitutes.
It should be noted that in the limit when the products really are homogeneous, there is no index number prob- lem, and the price ‘‘index’’ is given by the ratio of the unit values in the two periods, as explained later.

It may be conjectured that the average cross-elasticity is likely to be closer to unity than zero for most elementary aggregates so that, in general, the Jevons index is likely to provide a closer approximation to the cost of living index than the Carli.
In this case, the Carli index must be viewed as having an upward bias.
9.35 The insight provided by the economic approach is that the Jevons index is likely to provide a closer approximation to the cost of living index for the ele- in most mentary aggregate than the Carli because, cases, a signiﬁcant amount of substitution is more likely than no substitution, especially as elementary aggregates should be deliberately constructed in such a way as to group together similar items that are close substitutes for each other.
9.36 The Jevons index does not imply, or assume, that expenditure shares remain constant.

Obviously, the Jevons can be calculated whatever changes do, or do not occur in the expenditure shares in practice.

What the economic approach shows is that if the expenditure shares remain constant (or roughly constant), then the Jevons index can be expected to provide a good estimate of the underlying cost of living index.
Similarly, if the relative quantities remain constant, then the Carli index can be expected to provide a good estimate, but the Carli does not actually imply that quantities remain ﬁxed.

9.37 It may be concluded that, on the basis of the economic approach as well as the axiomatic approach, the Jevons emerges as the preferred index in general, although there may be cases in which little or no sub- stitution takes place within the elementary aggregate and the Carli might be preferred.
The index compiler must make a judgement on the basis of the nature of the products actually included in the elementary aggregate.

9.38 Before leaving this topic, it should be noted that it has thrown light on some of the sampling prop- erties of the elementary indices.
If the products in the sample are selected with probabilities proportional to expenditures in the price reference period: – the sample (unweighted) Carli index provides an unbiased estimate of the population Laspeyres; – the sample (unweighted) Jevons index provides an unbiased estimate of the population geometric Laspeyres.

These results hold irrespective of the underlying cost of living index.
Chain versus direct indices for elementary aggregates 9.39 In a direct elementary index, the prices of the current period are compared directly with those of the price reference period.

In a chain index, prices in each period are compared with those in the previous period, the resulting short-term indices being chained together to obtain the long-term index, as illustrated in Table 9.1.
9.40 Provided that prices are recorded for the same set of items in every period, as in Table 9.1, any index formula deﬁned as the ratio of the average prices will be transitive: that is, the same result is obtained whether the index is calculated as a direct index or as a chain index.

In a chain index, successive numerators and denominators will cancel out, leaving only the average price in the last period divided by the average price in the reference period, which is the same as the direct index.
Both the Dutot and the Jevons indices are therefore transitive.

As already noted, however, a chain Carli index is not tran- sitive and should not be used because of its upward bias.

Nevertheless, the direct Carli remains an option.


9.41 Although the chain and direct versions of the Dutot and Jevons indices are identical when there are no breaks in the series for the individual items, they oﬀer diﬀerent ways of dealing with new and disappearing CALCULATING CONSUMER PRICE INDICES IN PRACTICE items, missing prices and quality adjustments.
In prac- tice, products continually have to be dropped from the index and new ones included, in which case the direct and the chain indices may diﬀer if the imputations for missing prices are made diﬀerently.

9.42 When a replacement item has to be included in a direct index, it will often be necessary to estimate the price of the new item in the price reference period, which may be some time in the past.
The same happens if, as a result of an update of the sample, new items have to be linked into the index.

Assuming that no information exists on the price of the replacement item in the price reference period, it will be necessary to estimate it using price ratios calculated for the items that remain in the elementary aggregate, a subset of these items or some other indicator.

However, the direct approach should only be used for a limited period of time.
Otherwise, most of the reference prices would end up being imputed, which would be an undesirable outcome.

This eﬀectively rules out the use of the Carli index over a long period of time, as the Carli can only be used in its direct form anyway, being unacceptable when chained.

This implies that, in practice, the direct Carli may be used only if the overall index is chain linked annually, or at intervals of two or three years.
9.43 In a chain index, if an item becomes perma- nently missing, a replacement item can be linked into the index as part of the ongoing index calculation by including the item in the monthly index as soon as prices for two successive months are obtained.

Similarly, if the sample is updated and new products have to be linked into the index, this will require successive old and new prices for the present and the preceding months.
For a chain index, however, the missing observation will have an impact on the index for two months, since the missing observation is part of two links in the chain.

This is not the case for a direct index, where a single, non-estimated missing observation will only have an impact on the index in the current period.
For example, for a com- parison between periods 0 and 3, a missing price of an item in period 2 means that the chain index excludes the item for the last link of the index in periods 2 and 3, while the direct index includes it in period 3 since a direct index will be based on items whose prices are available in periods 0 and 3.
In general, however, the use of a chain index can make the estimation of missing prices and the introduction of replacements easier from a computa- tional point of view, whereas it may be inferred that a direct index will limit the usefulness of overlap methods for dealing with missing observations.

9.44 The direct and the chain approaches also pro- duce diﬀerent by-products that may be used for moni- toring price data.
For each elementary aggregate, a chain index approach gives the latest monthly price change, which can be useful for both data editing and imputation of missing prices.

By the same token, how- ever, a direct index derives average price levels for each elementary aggregate in each period, and this informa- tion may be a useful by-product.
Nevertheless, because the availability of cheap computing power and of spread- sheets allows such by-products to be calculated whether a direct or a chained approach is applied, the choice of 159 CONSUMER PRICE INDEX MANUAL: THEORY AND PRACTICE formula should not be dictated by considerations regarding by-products.
Consistency in aggregation 9.45 Consistency in aggregation means that if an index is calculated stepwise by aggregating lower-level indices to obtain indices at progressively higher levels of aggregation, the same overall result should be obtained as if the calculation had been made in one step.

For presentational purposes this is an advantage.


If the ele- mentary aggregates are calculated using one formula and the elementary aggregates are averaged to obtain the higher-level indices using another formula, the resulting CPI is not consistent in aggregation.
It may be argued, however, that consistency in aggregation is not necessar- ily an important or even appropriate criterion, or that it is unachievable when the amount of information avail- able on quantities and expenditures is not the same at the diﬀerent levels of aggregation.

In addition, there may be diﬀerent degrees of substitution within elemen- tary aggregates as compared to the degree of substitution between products in diﬀerent elementary aggregates.
9.46 As noted earlier, the Carli index would be consistent in aggregation with the Laspeyres index if the items were to be selected with probabilities proportional to expenditures in the reference period.

This is typically not the case.

The Dutot and the Jevons indices are also not consistent in aggregation with a higher-level Las- peyres.
As explained below, however, the CPIs actually calculated by statistical oﬃces are usually not true Las- peyres indices anyway, even though they may be based on ﬁxed baskets of goods and services.

As also noted earlier, if the higher-level index were to be deﬁned as a geometric Laspeyres, consistency in aggregation could be achieved by using the Jevons index for the elementary indices at the lower level, provided that the individual items are sampled with probabilities proportional to expenditures.
Although unfamiliar, a geometric Las- peyres has desirable properties from an economic point of view and is considered again later.

Missing price observations 9.47 The price of an item may fail to be collected in some period either because the item is missing tem- porarily or because it has permanently disappeared.

The two classes of missing prices require diﬀerent treatment.
Temporary unavailability may occur for seasonal items (particularly for fruit, vegetables and clothing), because of supply shortages or possibly because of some collec- tion diﬃculty (say, an outlet was closed or a price col- lector was ill).

The treatment of seasonal items raises a number of particular problems.

These are dealt with in Chapter 22 and will not be discussed here.

9.48 The treatment of temporarily missing prices.
In the case of temporarily missing observations for non- seasonal items, one of four actions may be taken: – omit the item for which the price is missing so that a matched sample is maintained (like is compared with like) even though the sample is depleted; – carry forward the last observed price; 160 – impute the missing price by the average price change for the prices that are available in the elementary aggregate; – impute the missing price by the price change for a particular comparable item from another similar outlet.
9.49 Omitting an observation from the calculation of an elementary index is equivalent to assuming that the price would have moved in the same way as the average of the prices of the items that remain included in the index.

Omitting an observation changes the implicit weights attached to the other prices in the elementary aggregate.

9.50 Carrying forward the last observed price should be avoided wherever possible and is acceptable only for a very limited number of periods.
Special care needs to be taken in periods of high inﬂation or when markets are changing rapidly as a result of a high rate of innovation and product turnover.

While simple to apply, carrying forward the last observed price biases the resulting index towards zero change.
In addition, there is likely to be a compensating step-change in the index when the price of the missing item is recorded again, which will be wrongly missed by a chain index, but will be included in a direct index to return the index to its proper value.

The adverse eﬀect on the index will be increasingly severe if the item remains unpriced for some length of time.

In general, to carry forward is not an acceptable procedure or solution to the problem.
9.51 Imputation of the missing price by the average change of the available prices may be applied for ele- mentary aggregates where the prices can be expected to move in the same direction.

The imputation can be made using all of the remaining prices in the elementary aggregate.
As already noted, this is numerically equiva- lent to omitting the item for the immediate period, but it is useful to make the imputation so that if the price becomes available again in a later period the sample size is not reduced in that period.

In some cases, depending on the homogeneity of the elementary aggregate, it may be preferable to use only a subset of items from the ele- mentary aggregate to estimate the missing price.
In some instances, this may even be a single comparable item from a similar type of outlet whose price change can be expected to be similar to the missing one.

9.52 Table 9.2 illustrates the calculation of the price index for an elementary aggregate consisting of three items where one of the prices is missing in March.

Section (a) of Table 9.2 shows the indices where the missing price has been omitted from the calculation.
The direct indices are therefore calculated on the basis of items A, B and C for all months except March, where they are calculated on the basis of items B and C only.

The chained indices are calculated on the basis of all three prices from Jan- uary to February and from April to May.

From Feb- ruary to March and from March to April the monthly indices are calculated on the basis of items B and C only.
9.53 For both the Dutot and the Jevons indices, the direct and chain indices now diﬀer from March onwards.
The ﬁrst link in the chain index (January to February) is the same as the direct index, so the two indices are CALCULATING CONSUMER PRICE INDICES IN PRACTICE Table 9.2 Imputation of temporarily missing prices Item A Item B Item C January February 6.00 7.00 2.00 5.00 8.00 3.00 (a) Omit missing prices from the index calculation Carli index – the arithmetic mean of price ratios Direct index Dutot index – the ratio of arithmetic mean prices Month-to-month index Chained month-to-month index Direct index Jevons index – the ratio of geometric mean prices = geometric mean of price ratios Month-to-month index Chained month-to-month index Direct index 112.62 112.62 112.62 100.00 100.00 100.00 106.67 106.67 106.67 100.00 100.00 100.00 115.87 100.00 (b) Imputation Carli index – the arithmetic mean of price ratios Impute price for item A in March as 5 (9/8 +4/3)/2 = 6.15 Direct index Dutot index – the ratio of arithmetic mean prices Impute price for item A in March as 5 ((9 +4)/(8 +3)) = 5.91 Month-to-month index Chained month-to-month index Direct index Jevons index – the ratio of geometric mean prices = geometric mean of price ratios Impute price for item A in March as 5 (9/8) (4/3)0.5 = 6.12 Month-to-month index Chained month-to-month index Direct index 100.00 100.00 100.00 112.62 112.62 112.62 106.67 106.67 106.67 100.00 100.00 100.00 115.87 100.00 March Prices 9.00 4.00 164.29 118.18 126.06 144.44 122.47 137.94 160.36 April May 7.00 8.00 3.00 126.98 84.62 106.67 120.00 81.65 112.62 125.99
identical numerically.
The direct index for March com- pletely ignores the price decrease of item A between January and February, while this is taken into account in the chain index.

As a result, the direct index is higher than the chain index for March.

On the other hand, in April and May, when all prices are again available, the direct index captures the price development, whereas the chain index fails to track the development in the prices.
In section (b) of Table 9.2 the missing price for item A in March is imputed by the average price change of the remaining items from February to March.

While the index may be calculated as a direct index, comparing the prices of the present period with the reference period prices, the imputation of missing prices should be made on the basis of the average price change from the pre- ceding to the present period, as shown in the table.
Imputation on the basis of the average price change from the base period to the present period should not be used as it ignores the information about the price change of the missing item that has already been included in the index.

The treatment of imputations is discussed in more detail in Chapter 7.

9.55 Treatment of items that have permanently dis- appeared and their replacements.

Items may disappear permanently for a variety of reasons.
The item may disappear from the market because new items have been introduced or the outlets from which the price has been collected have stopped selling the product.

Where prod- ucts disappear permanently, a replacement product has to be sampled and included in the index.
The replace- ment product should ideally be one that accounts for a signiﬁcant proportion of sales, is likely to continue to be sold for some time, and is likely to be representative of the sampled price changes of the market that the old product covered.

9.56 The timing of the introduction of replacement items is important.

Many new products are initially sold at high prices which then gradually drop over time, especially as the volume of sales increases.
Alternatively, some products may be introduced at artiﬁcially low prices to stimulate demand.

In such cases, delaying the introduction of a new or replacement item until a large volume of sales is achieved may miss some systematic price changes that ought to be captured by CPIs.
It may be desirable to try to avoid forced replacements caused when products disappear completely from the market, and to try to introduce replacements when sales of the items they replace are falling away, but before they cease altogether.

9.57 Table 9.3 shows an example where item A dis- appears after March and item D is included as a replace- ment from April onwards.

Items A and D are not available on the market at the same time and their price series do not overlap.
9.58 To include the new item in the index from April onwards, an imputed price needs to be calculated either for the base period (January) if a direct index is being calculated, or for the preceding period (March) if a chain index is calculated.

In both cases, the imputation method ensures that the inclusion of the new item does not, in itself, aﬀect the index.
In the case of a chain index, imputing the missing price by the average change of the available prices gives the same result as if the item is simply omitted from the index calculation until it has 161 CONSUMER PRICE INDEX MANUAL: THEORY AND PRACTICE Table 9.3 Disappearing items and their replacements with no overlapping prices Item A Item B Item C Item D January February 6.00 3.00 7.00 7.00 2.00 8.00 March Prices 5.00 4.00 9.00 April May 5.00 10.00 9.00 6.00 9.00 8.00 100.00 (a) Imputation Carli index – the arithmetic mean of price ratios Impute price for item D in January as 9/((5/3 + 10/7) 0.5) = 5.82 Direct index Dutot index – the ratio of arithmetic mean prices Impute price for item D in March as 9/((5 + 10)/(4 +9)) = 7.80 Month-to-month index Chained month-to-month index Impute price for item D in January as 9/((5 + 10)/(3 +7)) = 6.00 Direct index Jevons index – the ratio of geometric mean prices = geometric mean of price ratios Impute price for item D in March as 9/((5/4 10/9)0.5) = 7.64 Month-to-month index Chained month-to-month index Impute price for item D in January as 9/((5/3 10/7)0.5) = 5.83 Direct index 100.00 100.00 106.25 106.25 100.00 100.00 96.15 96.15 106.25 100.00 100.00 99.21 96.15 (b) Omit missing prices Dutot index – the ratio of arithmetic mean prices Month-to-month index Chained month-to-month index Jevons index – the ratio of geometric mean prices = geometric mean of price ratios Monthly index Chained month-to-month index 100.00 100.00 100.00 100.00 106.25 106.25
index been priced in two successive periods.
This allows the chain index to be compiled by simply chaining the month-to-month index between periods t1 and t, based on the matched set of prices in those two periods, onto the value of the chain index for period t1.

In the example, no further imputation is required after April, and the subsequent movement of the index is unaﬀected by the imputed price change between March and April.
In the case of a direct index, however, an impu- ted price is always required for the reference period in order to include a new item.

In the example, the price of the new item in each month after April still has to be compared with the imputed price for January.

As already noted, to prevent a situation in which most of the refer- ence period prices end up being imputed, the direct approach should only be used for a limited period of time.
9.60 The situation is somewhat simpler when there is an overlap month in which prices are collected for both the disappearing and the replacement item.

In this case, it is possible to link the price series for the new item to the price series for the old item that it replaces.

Linking with overlapping prices involves making an implicit adjustment for the diﬀerence in quality between the two items, as it assumes that the relative prices of the new and old item reﬂect their relative qualities.
For perfect or nearly perfect markets this may be a valid assumption, but for certain markets and products it may not be so reasonable.

The question of when to use overlapping prices is dealt with in detail in Chapter 7.

The overlap method is illustrated in Table 9.4.

9.61 In the example in Table 9.4, overlapping prices are obtained for items A and D in March.

Their relative 162 prices suggest that one unit of item D is worth two units of item A.
If the index is calculated as a direct Carli, the January base period price for item D can be imputed by dividing the price of item A in January by the price ratio of items A and D in March.

9.62 A monthly chain index of arithmetic mean pri- ces will be based on the prices of items A, B and C until March, and from April onwards on the prices of items B, C and D. The replacement item is not included until prices for two successive periods are obtained.
Thus, the monthly chain index has the advantage that it is not necessary to carry out any explicit imputation of a reference price for the new item.

9.63 If a direct index is calculated deﬁned as the ratio of the arithmetic mean prices, the price of the new item needs to be adjusted by the price ratio of A and D in March in every subsequent month, which complicates computation.

Alternatively, a reference period price of item D for January may be imputed.
This, however, results in a diﬀerent index because the price ratios are implicitly weighted by the relative base period prices in the Dutot index, which is not the case for the Carli or the Jevons indices.

For the Jevons index, all three methods give the same result, which is an additional advantage of this approach.

Other formulae for elementary price indices 9.64 A number of other formulae have been sug- gested for the price indices for elementary aggregates.
Table 9.4 Disappearing and replacement items with overlapping prices January February March April May CALCULATING CONSUMER PRICE INDICES IN PRACTICE 100.00 6.00 3.00 7.00 7.00 2.00 8.00 Item A Item B Item C Item D Carli index – the arithmetic mean of price ratios Impute price for item D in January as 6/(5/10) =12.00 Direct index Dutot index – the ratio of arithmetic mean prices Chain the monthly indices based on matched prices Month-to-month index Chained month-to-month index Divide item D’s price in April and May by 10/5 =2 and use item A’s price in January as base price Direct index Impute price for item D in January as 6/(5/10) =12.00 Direct index Jevons index – the ratio of geometric mean prices = geometric mean of price ratios Chain the monthly indices based on matched prices Month-to-month index Chained month-to-month index Divide item D’s price in April and May with 10/5 =2 and use item A’s price in January as base price Direct index Impute price for item D in January as 6/(5/10) =12.00 Direct index 100.00 100.00 100.00 100.00 106.25 106.25 96.15 96.15 106.25 100.00 100.00 106.25 100.00 100.00 96.15 99.21 96.15 Prices The most important are presented below and discussed further in Chapter 20.
9.65 Laspeyres and geometric Laspeyres indices.

The Carli, Dutot and Jevons indices are all calculated without the use of explicit weights.

However, as already mentioned, in certain cases there may be weight- ing information that could be exploited in the calcula- tion of the elementary price indices.
If the reference period expenditures for all the individual items within an elementary aggregate, or estimates thereof, were to be available, the elementary price index could itself be calculated as a Laspeyres price index, or as a geo- metric Laspeyres.

The Laspeyres price index is deﬁned as: La =Pw0 I 0:t i pt i p0 i , Pw0 i =1 (9.4) 0, are the expenditure shares for the where the weights, wi individual items in the reference period.
If all the weights were equal, the formula (9.4) would reduce to the Carli index.

If the weights were proportional to the prices in the reference period, the formula (9.4) would reduce to the Dutot index.

9.66 The geometric version of the Laspeyres index is deﬁned as: wi JW =Q pt I0:t i p0 i 0 = i Q ( pt Q ( p0 i)w0 i Þw0 i , Pw0 i =1 (9.5) 0, are again the expenditure shares where the weights, wi in the reference period.
When the weights are all equal, the formula (9.5) reduces to the Jevons index.

9.67 Some alternative index formulae.

Another widely used type of average is the harmonic mean.

In the present context, there are two possible versions: either the harmonic mean of price ratios or the ratio of harmonic mean prices.
The harmonic mean of price relatives, or ratios, is deﬁned as: I 0:t HR= 1 P p0 i pt i 1 n The ratio of harmonic mean prices is deﬁned as: I 0:t RH= P n p0 i P n pt i (9.6) (9.7) Formula (9.7), like the Dutot index, fails the commen- surability test and would only be an acceptable possi- bility when the items are all fairly homogeneous.

Neither formula appears to be used much in practice, perhaps because the harmonic mean is not a familiar concept and would not be easy to explain to users.
Nevertheless, at an aggregate level, the widely used Paasche index is a weighted harmonic average.

9.68 The ranking of the three common types of mean is always arithmetic geometric harmonic.

It is shown in Chapter 20 that, in practice, the Carli index (the arithmetic mean of the relatives) is likely to exceed the Jevons index (the geometric mean) by roughly the same amount that the Jevons exceeds the harmonic mean given by the formula (9.6).
The harmonic mean of the price relatives has the same kinds of axiomatic properties as the Carli index, but with opposite tendencies and biases.

It fails the transitivity, time reversal and price bouncing tests.
As it can be viewed conceptually as the complement, or rough mirror image, of the Carli index, it has been argued that a suitable elementary index would be provided by a geometric mean of the two, in the same way that, at an aggregate level, a geometric mean is taken of the Laspeyres and Paasche indices to obtain the Fisher 163 CONSUMER PRICE INDEX MANUAL: THEORY AND PRACTICE index.
Such an index has been proposed by Carruthers, Sellwood and Ward (1980) and Dale´ n (1992), namely: CSWD=(I 0:t I 0:t C I 0:t HR)1=2 (9.8) ICSWD is shown in Chapter 20 to have very good axiom- atic properties, although not quite as good as the Jevons index, which is transitive, whereas the ICSWD is not.

It can, however, be shown to be approximately transitive, and it has been observed empirically to be very close to the Jevons index.
9.69 In recent years, attention has focused on for- mulae that can take account of the substitution that may take place within an elementary aggregate.

As already explained, the Carli and the Jevons indices may be expected to approximate to the cost of living index if the cross-elasticities of substitution are close to 0 and 1, respectively, on average.
A more ﬂexible formula that allows for diﬀerent elasticities of substitution is the unweighted Lloyd–Moulton (LM) index: " LM= P 1 I 0:t n 1s Pt i P0 i # 1 1s (9.9) where s is the elasticity of substitution.

The Carli and the Jevons indices can be viewed as special cases of the LM in which s=0 and s=1.

The advantage of the LM formula is that s is unrestricted.
Provided a satisfactory estimate can be made of s, the resulting elementary price index is likely to approximate to the underlying cost of living index.

The LM index reduces ‘‘substitution bias’’ when the objective is to estimate the cost of living index.

The diﬃculty is the need to estimate elasticities of sub- stitution, a task that will require substantial develop- ment and maintenance work.

The formula is described in more detail in Chapter 17.
Unit value indices 9.70 The unit value index is simple in form.

The unit value in each period is calculated by dividing total expenditure on some product by the related total quan- tity.

It is clear that the quantities must be strictly additive in an economic sense, which implies that they should relate to a single homogeneous product.

The unit value index is then deﬁned as the ratio of unit values in the current period to that in the reference period.
It is not a price index as normally understood, as it is essentially a measure of the change in the average price of a single product when that product is sold at diﬀerent prices to diﬀerent consumers, perhaps at diﬀerent times within the same period.

Unit values, and unit value indices, should not be calculated for sets of heterogeneous products.
9.71 Unit values do play an important part in the process of calculating an elementary price index, as they are the appropriate average prices that need to be entered into an elementary price index.

Usually, prices are sam- pled at a particular time or period each month, and each price is assumed to be representative of the average price of that item in that period.

In practice, this assumption may not hold.


In this case, it is necessary to estimate the unit value for each item, even though this will inevitably be more costly.

Thus, having speciﬁed the item to be 164 priced in a particular store, data should be collected on both the value of the total sales in a particular month and the total quantities sold in order to derive a unit value to be used as the price input into an elementary aggregate formula.
It is particularly important to do this if the item is sold at a sale price for part of the period and at the regular price in the rest of the period.

Under these con- ditions, neither the sale price nor the regular price is likely to be representative of the average price at which the item has been sold or the price change between peri- ods.

The unit value over the whole month should be used.
With the possibility of collecting more and more data from electronic points of sale, such procedures may be increasingly used.

It should be stressed, however, that the item speciﬁcations must remain constant through time.

Changes in the item speciﬁcations could lead to unit value changes that reﬂect quantity or quality changes, and should not be part of price changes.
Formulae applicable to scanner data 9.72 Scanner data obtained from electronic points of sale are becoming an increasingly important source of data for CPI compilation.

Their main advantage is that the number of price observations can be enormously increased and that both price and quantity information is available in real time.

There are, however, many practical considerations to be taken into consideration which are discussed in other chapters of this manual.
9.73 Access to detailed and comprehensive quantity and expenditure information within an elementary aggregate means that there are no constraints on the type of index number that may be employed.

Not only Laspeyres and Paasche but superlative indices such as Fisher and To¨ rnqvist may be envisaged.

As noted at the beginning of is preferable to this chapter, introduce weighting information as it becomes available rather than continuing to rely on simple unweighted indices such as Carli and Jevons.
Advances in technol- ogy, both in the retail outlets themselves and in the computing power available to statistical oﬃces, suggest that traditional elementary price indices may eventually be replaced by superlative indices, at least for some ele- mentary aggregates in some countries.

The methodology must be kept under review in the light of the resources available.

it The calculation of higher-level indices 9.74 A statistical oﬃce must have some target index at which to aim.
Statistical oﬃces have to consider what kind of index they would choose to calculate in the ideal hypothetical situation in which they had complete information about prices and quantities in both time periods compared.

If the CPI is meant to be a cost of living index, then a superlative index such as a Fisher, Walsh or Tornqvist–Theil would have to serve as the theoretical target, as a superlative index may be expected to approximate to the underlying cost of living index.
9.75 Many countries do not aim to calculate a cost of living index and prefer the concept of a basket index.

A basket index is one that measures the change in the total value of a given basket of goods and services between two time periods.

This general category of index is described here as a Lowe index (see Chapters 1 and 15).

The meaning of a Lowe index is clear and can be easily explained to users, these being important considerations for many statistical oﬃces.
It should be noted that, in general, there is no necessity for the basket to be the actual basket in one or other of the two periods com- pared.

If the theoretical target index is to be a basket or Lowe index, the preferred basket might be one that attaches equal importance to the baskets in both periods; for example, the Walsh index.

The quantities that make up the basket in the Walsh index are the geometric means of the quantities in the two periods.
Thus, the same kind of index may emerge as the theoretical target in both the basket and the cost of living approaches.

In practice, a statistical oﬃce may prefer to designate a basket index that uses the actual basket in the earlier of the two periods as its target index on grounds of simplicity and practi- cality.

In other words, the Laspeyres index may be the target index.

9.76 The theoretical target index is a matter of choice.
In practice, it is likely to be either a Laspeyres or some superlative index.

Even when the target index is the Laspeyres, there may a considerable gap between what is actually calculated and what the statistical oﬃce con- siders to be its target.

It is now necessary to consider what statistical oﬃces tend to do in practice.
Consumer price indices as weighted averages of elementary indices 9.77 A higher-level index is an index for some expenditure aggregate above the level of an elementary aggregate, including the overall CPI itself.

The inputs into the calculation of the higher-level indices are: – the elementary price indices; – weights derived from the values of elementary aggre- gates in some earlier year or years.
9.78 The higher-level indices are calculated simply as weighted arithmetic averages of the elementary price indices.

This general category of index is described here as a Young index after another nineteenth-century index number pioneer who advocated this type of index.

9.79 The weights typically remain ﬁxed for a sequence of at least 12 months.
Some countries revise their weights at the beginning of each year in order to try to approximate as closely as possible to current consump- tion patterns, but many countries continue to use the same weights for several years.

The weights may be changed only every ﬁve years or so.

The use of ﬁxed weights has the considerable practical advantage that the index can make repeated use of the same weights.

This saves both time and money.


Revising the weights can be both time-consuming and costly, especially if it requires new household expenditure surveys to be carried out.

9.80 The second stage of calculating a CPI does not involve individual prices or quantities.

Instead, a higher- level index is a Young index in which the elementary CALCULATING CONSUMER PRICE INDICES IN PRACTICE price indices are averaged using a set of pre-determined weights.
The formula can be written as follows: , Pwb I 0:t=Pwb i =1 (9.10) i I 0:t i where I 0:t denotes the overall CPI, or any higher-level b is the weight attached to index, from period 0 to t; wi each of the elementary price indices; and Ii is the corresponding elementary price index.

The elementary indices are identiﬁed by the subscript i, whereas the higher-level index carries no subscript.

As already noted, a higher-level index is any index, including the overall CPI, above the elementary aggregate level.
The weights are derived from expenditures in period b, which in practice has to precede period 0, the price reference period.

9.81 It is useful to recall that three kinds of reference 0:t period may be distinguished for CPI purposes: Weight reference period.

The period covered by the expenditure statistics used to calculate the weights.

Usually, the weight reference period is a year.

 Price reference period.

The period for which prices are used as denominators in the index calculation.
Index reference period.

The period for which the index is set to 100.

9.82 The three periods are generally diﬀerent.

For example, a CPI might have 1998 as the weight reference year, December 2002 as the price reference month and the year 2000 as the index reference period.

The weights typically refer to a whole year, or even two or three years, whereas the periods for which prices are com- pared are typically months or quarters.
The weights are usually estimated on the basis of an expenditure survey that was conducted some time before the price reference period.

For these reasons, the weight reference period and the price reference period are invariably separate periods in practice.

9.83 The index reference period is often a year; but it could be a month or some other period.
An index series may also be re-referenced to another period by simply dividing the series by the value of the index in that period, without changing the rate of change of the index.

The expression ‘‘base period’’ can mean any of the three reference periods and is ambiguous.

The expression ‘‘base period’’ should only be used when it is absolutely clear in context exactly which period is referred to.
9.84 Provided the elementary aggregate indices are calculated using a transitive formula such as the Jevons or Dutot, but not the Carli, and provided that there are no new or disappearing items from period 0 to t, equation (9.10) is equivalent to: I 0:t=Pwb i I 0:t1 i I t1:t i , Pwb i =1, (9.11) The advantage of this version of the index is that it allows the sampled products within the elementary price index from t1 to t to diﬀer from the sampled products in the periods from 0 to t1.
Hence, it allows replacement items and new items to be linked into the index from period t1 without the need to estimate a price for per- iod 0.

For example, if one of the sampled items in periods 0 and t1 is no longer available in period t, and the price 165 CONSUMER PRICE INDEX MANUAL: THEORY AND PRACTICE of a replacement product is available for t1 at t, the new replacement product can be included in the index using the overlap method.
A numerical example 9.85 Equation (9.10) applies at each level of aggre- gation.

The index is additive; that is, the overall index is the same whether calculated on the basis of the original elementary price indices or on the basis of the inter- mediate higher-level indices.

This facilitates the presen- tation of the index.

9.86 Table 9.5 illustrates the calculation of higher- level indices in the special case where the weight and the price reference period are identical, i.e.

b=0.
b=0.

The index consists of ﬁve elementary aggregate indices and two intermediate higher-level indices, G and H. The overall index and the higher-level indices are all calculated using equation (9.10).
Thus, for example, the overall index for April can be calculated from the two intermediate higher-level indices for April as: I Jan:Apr=0:6 103:92+0:4 101:79=103:06 or directly from the ﬁve elementary indices as: I Jan:Apr=0:2 108:75+0:25 100+0:15 104+0:1 107:14+0:3 100=103:06 Note from equation (9.11) that: I 0:t=Pwb i I 0:t1 i I 0:t I 0:t1 6¼ Pwb i I t1:t i I t1:t i 6¼ I 0:t1Pwb i I t1:t i ) (9.12) This shows that if the month-to-month indices are b, the resulting index averaged using the ﬁxed weights wi is not equal to the month-to-month higher-level index.
As explained later, in order to be able to obtain the month-to-month higher-level index, the weights applied to the month-to-month indices need to be updated to reﬂect the eﬀects of the price changes that have taken place since January.

Table 9.5 The aggregation of elementary price indices Young and Lowe indices 9.87 It is useful to clarify the relationship between Lowe and Young indices.
As already noted, when sta- tistical oﬃces explain their CPIs to users they often describe them as Lowe indices, which measure the change over time in the value of a ﬁxed basket of goods and services.

But when they calculate their CPIs, the formula they actually use is that of a Young index.
The rela- tionship between the two indices is given in equation (9.13), where ILo is the Lowe index and IYo is the Young index: ILo= P pt P p0 jqb j j qb j = where wj= jqb j j qb j P pt P pb j qb po j P p0 j qb j , P p0 P pb j qb j j qb j =Pwj !

pt j p0 j =IYo (9.13) b, the individual quantities in the weight The values qj reference period b, make up the basket.

Assume initially that the weight reference period b has the same dura- tion as that of the two periods 0 and t that are being compared.
It can be seen from the relationship (9.13) that: the Lowe index is equal to a Young index in which the weights are hybrid value shares obtained by revaluing the values qb, the quantities in the weight reference period b, at the prices of the price reference month 0; the Lowe index can be expressed as the ratio of the two Laspeyres indices for periods t and 0, respectively, based on month b; the Lowe index reduces to the Laspeyres index when b = 0, and to the Paasche index when b=t.
9.88 In practice, the situation is more complicated for actual CPIs because the duration of the reference period b is typically much longer than periods 0 and t. The weights wj usually refer to the expenditures over a period of a year, or longer, while the price reference period is usually a month in some later year.
For exam- ple, a monthly index may be compiled from January 2003 onwards with December 2002 as the price reference Weight January February March April May June 0.20 0.25 0.15 0.10 0.30 100.00 100.00 100.00 100.00 100.00 102.50 100.00 104.00 92.86 101.67 Month-to-month elementary price indices A B C D E Direct or chained monthly elementary price indices with January =100 A B C D E Total Higher-level indices G = A + B + C H = D + E Total month, but the latest available weights during the year 2003 may refer to the year 2000 or even some earlier year.
9.89 Conceptually, a typical CPI may be viewed as a Lowe index that measures the change from month to month in the total cost of an annual basket of goods and services that may date back several years before the price reference period.

Because it uses the ﬁxed basket of an earlier period, it is sometimes loosely described as a ‘‘Laspeyres-type index’’, but this description is unwar- ranted.
A true Laspeyres index would require the basket to be that consumed in the price reference month, whereas in most CPIs the basket not only refers to a diﬀerent period from the price reference month but to a period of a year or more.

When the weights are annual and the prices are monthly, it is not possible, even retro- spectively, to calculate a monthly Laspeyres price index.
9.90 As shown in Chapter 15, a Lowe index that uses quantities derived from an earlier period than the price reference period is likely to exceed the Laspeyres, and by a progressively larger amount, the further back in time the weight reference period is.

The Lowe index is likely to have an even greater upward bias than the Laspeyres as compared with some target superlative index or underlying cost of living index.
Inevitably, the quantities in any basket index become increasingly out of date and irrelevant the further back in time the period to which they relate.

To minimize the result- ing bias, the weights should be updated as often as possible.

9.91 A statistical oﬃce may not wish to estimate a cost of living index and may prefer to choose some basket index as its target index.
In that case, if the the- oretically attractive Walsh index were to be selected as the target index, a Lowe index would have the same bias as just described, given that the Walsh index is also a superlative index.
Factoring the Young index 9.92 It is possible to calculate the change in a higher- level Young index between two consecutive periods, such as t1 and t, as a weighted average of the indivi- dual price indices between t1 and t provided that the weights are updated to take account of the price changes between the price reference period 0 and the previous period, t1.
This makes it possible to factor the formula (9.10) into the product of two component indices in the following way: I 0:t=I 0:t1Pwb(t1) where wb(t1) =wb i I t1:1 i i I 0:t1 i i .Pwb i I 0:t1 i : (9.14) I 0:t1 is the Young index for period t1.
The weight b(t1) is the original weight for elementary aggregate i wi price-updated by multiplying it by the elementary price index for i between 0 and t1, the adjusted weights being rescaled to sum to unity.

The price-updated weights are hybrid weights because they implicitly revalue the quantities of b at the prices of t1 instead CALCULATING CONSUMER PRICE INDICES IN PRACTICE of at the average prices of b.
Such hybrid weights do not measure the actual expenditure shares of any period.

9.93 The index for period t can thus be calculated by multiplying the already calculated index for t1 by a separate Young index between t1 and t with hybrid price-updated weights.

In eﬀect, the higher-level index is calculated as a chain index in which the index is moved forward period by period.
This method gives more ﬂex- ibility to introduce replacement items and makes it easier to monitor the movements of the recorded prices for errors, as month-to-month movements are smaller and less variable than the total changes since the base period.

9.94 Price-updating may also occur between the weight reference period and the price reference period, as explained in the next section.
Price-updating from the weight reference period to the price reference period 9.95 When the weight reference period b and the price reference period 0 are diﬀerent, as is normally the case, the statistical oﬃce has to decide whether or not to price-update the weights from b to 0.

In practice, the price-updated weights can be calculated by multiplying the original weights for period b by elementary indices measuring the price changes between periods b and 0 and rescaling to sum to unity.
9.96 The issues involved are best explained with the help of a numerical example.

In Table 9.6, the base period b is assumed to be the year 2000, so the weights are the expenditure shares in 2000.

In section (a) of Table 9.6, 2000 is also used as the price reference period.

In practice, however, weights based on 2000 cannot be introduced until some time after 2000 because of the time needed to collect and process the expenditure data.
In section (b) of Table 9.6, it is assumed that the 2000 weights are introduced in December 2002 and that this is also chosen as the new price reference base.

9.97 Note that it would be possible in December 2002 to calculate the indices based on 2000 shown in section (a) of the table, but it is decided to make December 2002 the price reference base.

This does not prevent the index with the December 2002 price reference period from being calculated backwards a few months into 2002, if desired.
9.98 The statistical oﬃce compiling the index has two options at the time the new index is introduced.

It has to decide whether the weights in the new index should preserve the quantities in 2000 or the expenditures in 2000.

It cannot do both.

9.99 If it is decided to preserve the quantities, the resulting index is a basket index, or Lowe index, in which the quantities are those of the year 2000.
This implies that the movements of the index must be identical with those of the index based on 2000 shown in section (a) of the table.

In this case, if the index is to be presented as a weighted average of the elementary price indices with December 2002 as the price reference period, the expenditure weights for 2000 have to be price-updated to December 2002.
This is illustrated in section (b) of Table 9.6, where the updated weights are obtained by multi- plying the original weights for 2000 in section (a) of the 167 CONSUMER PRICE INDEX MANUAL: THEORY AND PRACTICE Table 9.6 Price-updating of weights between the weight and price reference periods Weight 2000 Nov. 02 Dec. 02 Jan. 03 Feb. 03 Mar.
03 (a) Index with 2000 as weight and price reference period Elementary price indices 99.00 108.00 106.00 104.00 103.00 Higher-level indices 104.50 103.25 104.00 A B C D E w00 0.20 0.25 0.15 0.10 0.30 100.00 100.00 100.00 100.00 100.00 98.00 106.00 104.00 101.00 102.00 G = A + B + C H = D + E Total (b) Index re-referenced to December 2002 and weights price-updated to December 2002 100.00 100.00 100.00 102.83 101.75 102.40 0.60 0.40 A B C D E G = A + B + C H = D + E Total Rescaled to 2000 = 100 w00(Dec02) 0.190 0.260 0.153 0.100 0.297 0.603 0.397 101.01 92.59 94.34 96.15 97.09 95.69 96.85 96.15 100.00 Elementary price indices 98.99 98.15 98.11 97.12 99.03 98.41 98.55 98.46 102.40 100.00 100.00 100.00 100.00 100.00 Higher-level indices table by the price indices for the elementary aggregates between 2000 and December 2002, and then rescaling the results to sum to unity.
These are the weights labelled w00(Dec02) in the table.

9.100 The indices with price-updated weights in section (b) of Table 9.6 are Lowe indices in which b= 2000 and 0=December 2002.

These indices can be expressed as ratios of the indices in the upper part of the table.
For example, the overall Lowe index for March 2003 with December 2002 as its price reference base, namely 101.97, is the ratio of the index for March 2003 based on 2000 shown in section (a) of the table, namely 106.05, divided by the index for December 2002 based on 2000, namely 104.00.

Thus, the price-updating preserves the movements of the indices in section (a) of the table while shifting the price reference period to December 2002.
9.101 Alternatively, it could be decided to calculate a series of Young indices using the expenditure weights from 2000 as they stand without updating.

If the expenditure shares were actually to remain constant, the quantities would have had to move inversely with the prices between 2000 and December 2002.

The quantities that make up the basket for the new Young index could not be the same as those of 2000.
The movements of this index would have to be slightly diﬀerent from those of the price-updated index.

9.102 The issue is whether to stick with the known quantities of the weight reference period 2000, which are the latest for which ﬁrm data have been collected, or to stick with the known expenditure shares of the weight reference period.

If the oﬃcial objective is to measure a Lowe index that uses a ﬁxed basket, the issue is decided and the statistical oﬃce is obliged to price-update.
On the other hand, some statistical oﬃces may have to decide for themselves which option to adopt.

9.103 Updating the prices without updating the quantities does not imply that the resulting expenditure weights are necessarily more up to date.

When there is a strong inverse relation between movements of price and quantities, price-updating on its own could produce perverse results.

For example, the prices of computers have been declining rapidly in recent years.
If the quan- tities are held ﬁxed while the prices are updated, the resulting expenditures on computers would also decline rapidly.

In practice, however, the share of expendi- tures on computers might be actually be rising because of a very rapid increase in quantities of computers purchased.
9.104 When there are rapid changes taking place in relative quantities as well as relative prices, statistical oﬃces are eﬀectively obliged to change their expenditure weights more frequently, even if this means conducting more frequent expenditure surveys.

Price-updating on its own cannot cope with this situation.

The expenditure weights have to be updated with respect to their quan- tities as well as their prices, which, in eﬀect, implies collecting new expenditure data.
The introduction of new weights and chain linking 9.105 From time to time, the weights for the ele- mentary aggregates have to be revised to ensure that they reﬂect current expenditure patterns and consumer behaviour.

When new weights are introduced, the price reference period for the new index can be the last period of the old index, the old and the new indices being linked together at this point.

The old and the new indices make a chain index.


9.106 The introduction of new weights is often a complex operation because it provides the opportunity 168 to introduce new items, new samples, new data sources, new compilation practices, new elementary aggregates, new higher-level indices or new classiﬁcations.

These tasks are often undertaken simultaneously at the time of reweighting to minimize overall disruption to the time series and any resulting inconvenience to users of the indices.
9.107 In many countries, reweighting and chaining are carried out about every ﬁve years, but some countries introduce new weights each year.

Chain indices do not have to be linked annually; the linking may be done less frequently.

The real issue is not whether to chain or not but how frequently to chain.

Reweighting is inevitable sooner or later, as the same weights cannot continue to be used for ever.
Whatever the time frame, sta- tistical oﬃces have to address the issue of chain linking sooner or later.

It is inevitable and a major task for index compilers.

9.108 Frequency of reweighting.

It is reasonable to continue to use the same set of elementary aggregate weights so long as consumption patterns at the elemen- tary aggregate level remain fairly stable.

Over time, consumers will tend to substitute away from products of which the prices have increased relatively.
Thus, in gen- eral, movements in prices and quantities tend to be inversely related.

This kind of substitution behaviour on the part of consumers implies that a Lowe index based on the ﬁxed basket of an earlier period will tend to have an upward bias compared with a basket index using up-to-date weights.

9.109 Another reason why consumption patterns change is that new products are continually being introduced on the market while others drop out.
Over the longer term, consumption patterns are also inﬂuenced by several other factors.

These include rising incomes and standards of living, demographic changes in the structure of the population, changes in technology, and changes in tastes and preferences.

9.110 There is wide consensus that regular updating of weights – at least every ﬁve years, and more often if there is evidence of rapid changes in consumption pat- terns – is a sensible and necessary practice.
The question of how often to change the weights and chain link the index is nevertheless not straightforward, as frequent linking can also have some disadvantages.

It can be costly to obtain new weights, especially if they require more frequent expenditure surveys.

Annual chaining has the advantage that changes (such as the inclusion of new goods) can be introduced on a regular basis, although every index needs some ongoing maintenance, whether annually chained or not.
9.111 Expenditures on certain types of products are strongly inﬂuenced by short-term ﬂuctuations in the economy.

For example, expenditures on cars, major durables, expensive luxuries, and so on, may change dras- tically from year to year.

In such cases, it may be pre- ferable to base the weight on an average of two or more years of expenditure.

9.112 The calculation of a chain index.


Assume that a series of ﬁxed weight Young indices has been calculated with period 0 as the price reference period and that in a subsequent period, k, a new set of weights has to be CALCULATING CONSUMER PRICE INDICES IN PRACTICE introduced in the index.
The new set of weights may, or may not, have been price-updated from the new weight reference period to period k. A chain index is then cal- culated as: I t1:t i I 0:t=I 0:kPwk =I 0:kPwk =I 0:kI k:t i I k:t1 i i I k:t i (9.15) 9.113 There are several important features of a chain index: The chain index formula allows weights to be up- dated, and facilitates the introduction of new items and sub-indices and the removal of obsolete ones.
In order to be able to link the old and the new series, an overlapping period (k) is needed in which the index has to be calculated using both the old and the new set of weights.

 A chain index may have two or more links.

Between each link period, the index may be calculated as a ﬁxed weight index using the formula (9.10), or indeed using any other index formula.

The link period may be a month or a year, provided the weights and indices refer to the same period.
Chaining is intended to ensure that the individual indices on all levels show the correct development through time.

 Chaining leads to non-additivity.

When the new series is chained onto the old, as in equation (9.15), the higher-level indices after the link cannot be obtained as weighted arithmetic averages of indivi- dual indices using the new weights.
If, on the other hand, the index reference period is changed and the index series prior to the link period is rescaled to the new index reference period, this series cannot be aggregated to higher-level indices by use of the new weights.

Such results need to be carefully explained and presented.

9.114 An example of the calculation of a chain index is presented in Table 9.7.

From 1998 to December 2002 the index is calculated with the year 1998 as weight and price reference period.
From December 2002 onwards, a new set of weights is introduced.

The weights may refer to the year 2000, for example, and may or may not have been price-updated to December 2002.

A new ﬁxed weight index series is then calculated with December 2002 as the price reference month.

Finally, the new index series is linked onto the old index with 1998=100 by multiplication to get a continuous index from 1998 to March 2003.
The chained higher-level indices in Table 9.7 are calculated as: I 00:t=I 98:Dec02Pw00(Dec 02) i I Dec 02:t i (9.16) Because of the lack of additivity, the overall chain index for March 2003 (129.07), for example, cannot be cal- culated as the weighted arithmetic mean of the chained higher-level indices G and H using the weights from December 2002.
169 CONSUMER PRICE INDEX MANUAL: THEORY AND PRACTICE Table 9.7 Calculation of a chain index Weight 1998 1998 Nov. 2002 Dec. 2002 Weight 2000 Dec. 2002 Jan. 2003 Feb. 2003 Mar.
2003 1998 = 100 December 2002 = 100 0.20 0.25 0.15 0.10 0.30 100.00 100.00 100.00 100.00 100.00 100.00 Elementary price indices A B C D E Total Higher-level indices 100.00 G = A + B + C 100.00 H = D + E Total 100.00 Chaining of higher-level indices to 1998 = 100 100.00 G = A + B + C 100.00 H = D + E 100.00 Total The introduction of new elementary aggregates.
First, consider the situation in which new weights are introduced and the index is chain linked in December 2002.

The overall coverage of the CPI is assumed to remain the same, but certain items have increased suﬃciently in importance to merit recognition as new elementary aggregates.

Possible examples are the introduction of new elementary aggregates for mobile telephones or Internet access.

9.116 Consider the calculation of the new index from December 2002 onwards, the new price reference period.
The calculation of the new index presents no special problems and can be carried out using formula (9.10).

However, if the weights are price-updated from, say, 2000 to December 2002, diﬃculties may arise because the elementary aggregate for mobile telephones did not exist prior to December 2002, so there is no price index with which to price-update the weight for mobile telephones.
Prices for mobile telephones may have been recorded prior to December 2002, possibly within another ele- mentary aggregate (communications equipment), so it may be possible to construct a price series which can be used for price-updating.

Otherwise, price information from other sources, such as purchasing power parity (PPP) surveys, business statistics, or industry sources, may have to be used.
If no information is available, then movements in the price indices for similar elementary aggregates may be used as proxies for price-updating.

9.117 The inclusion of a new elementary aggregate means that the next higher-level index contains a diﬀer- ent number of elementary aggregates before and after the linking.

Therefore, the rate of change of the higher-level index whose composition has changed may be diﬃcult to interpret.
However, failing to introduce new goods or services for this reason would result in an index that does not reﬂect the actual dynamic changes taking place in the economy.

If it is customary to revise the CPI backwards, then the prices of the new product and their weights might be introduced retrospectively.

If the CPI is not revised backwards, which is usually the case, there is little that can be done to improve the quality of the chain index.
In many cases, the addition of a single elementary aggregate is unlikely to have a signiﬁcant eﬀect on the 170 next higher-level index into which it enters.

If the addi- tion of an elementary aggregate is believed to have a signiﬁcant impact on the time series of the higher-level index, it may be necessary to discontinue the old series and commence a new higher-level index.

These decisions can only be made on a case-by-case basis.

9.118 The introduction of new higher-level indices.
It may be necessary to introduce a new higher-level index in the overall CPI.

This situation may occur if the coverage of the CPI is enlarged or the grouping of elementary aggregates is changed.

It then needs to be decided what the initial value of the new higher-level index should be when it is included in the calculation of the overall CPI.

Take as an example the situation in Table 9.7 and assume that a new higher-level index from January 2003 has to be included in the index.
The question is then what should be the December 2002 value to which the new higher-level index is to be linked.

There are two options: Estimate the value in December 2002 that the new higher-level index would have had with 1998 as the price reference period, and link the new series from January 2003 onwards on to this value.

This proce- dure will prevent any break in the index series.

 Use 100 in December 2002 as the starting point for the new higher-level index.
This simpliﬁes the problem from a calculation perspective, although there remains the diﬃculty of explaining the index break to users.

In any case, major changes such as those just described should, so far as possible, be made in connection with the regular reweighting and chaining in order to mini- mize disruptions to the index series.

9.119 A ﬁnal case to consider concerns classiﬁcation change.
For example, a country may decide to change from a national classiﬁcation to an international one, such as the Classiﬁcation of Individual Consumption according to Purpose (COICOP).

The changes in the composition of the aggregates within the CPI may then be so large that it is not meaningful to link them.

In such cases, it is recommended that the CPI with the new classiﬁcation should be calculated backwards for at least one year so that consistent annual rates of change can be calculated.
9.120 Partial reweighting.

The weights for the ele- mentary aggregates may be obtained from a variety of sources over a number of diﬀerent periods.

Conse- quently, it may not be possible to introduce all the new weighting information at the same time.

In some cases, it may be preferable to introduce new weights for some elementary aggregates as soon as possible after the information is received.

The introduction of new weights for a subset of the overall index is known as partial reweighting.
9.121 Partial reweighting has particular implications for the practice of price-updating the weights.

Weighting information may not be available for some elementary aggregates at the time of rebasing.

Thus, it may be necessary to consider price-updating not only the new weights, but also the old weights for those elementary aggregates for which no new weights are available.
The weights for the latter may have to be price-updated over a long period, which, for reasons given earlier, may give rise to serious problems if relative quantities have changed inversely to the relative price changes.

Data on both quantity and price changes should be sought before undertaking such updates.

The disadvantage of partial reweighting is that the implicit quantities belong to diﬀerent periods, so that the composition of the basket is obscure and not well deﬁned.
9.122 It may be concluded that the introduction of new weights and the linking of a new series to an old series is not diﬃcult in principle.

The diﬃculties arise in practice when trying to align weight and price reference periods and when deciding whether higher-level indices comprising diﬀerent elementary aggregates should be chained over time.
It is not possible for this manual to provide speciﬁc guidance on decisions such as these, but compilers should consider carefully the economic logic and statistical reliability of the resulting chained series and also the needs of users.

In order to facilitate the thought should be decision-making process, careful given to these issues in advance during the planning of a reweighting exercise, paying particular attention to which of the indices are to be published.

9.123 Long-term and short-term links.


Consider a long-term chain index in which the weights are changed annually.

In any given year, the current monthly indices are ﬁrst calculated using the latest set of available weights, which cannot be those of the current year.

However, when the weights for the year in question become available subsequently, the monthly indices can then be recalculated on the basis of the weights for that same year.
The resulting series can then be used in the long-term chain index, rather than the original indices ﬁrst published.

Thus, the movements of the long-term chain index from, say, any one December to the following December are based on weights of that same year, the weights being changed each December.

This method has been developed by the Central Statis- tical Oﬃce of Sweden, where it is applied in the calcu- lation of the CPI.
It is described in Swedish Consumer Price Index: A Handbook of Methods (Statistics Sweden, 2001).

9.124 Assume that each link runs from December to December.

The long-term index for month m of year Y CALCULATING CONSUMER PRICE INDICES IN PRACTICE with December of year 0 as index reference period is then calculated using the formula: I Dec0:mY = Y Y1 y=1 !

I Decy1:Decy I DecY1:mY ð9:17Þ =I Dec 0:Dec1I Dec1:Dec2 .

.

.
.

.

I DecY 2:DecY 1I DecY 1:mY In the actual Swedish practice, a factor scaling the index from December year 0 to the average of year 0 is mul- tiplied onto the right-hand side of the formula (9.19) to have a full year as the reference period.

The long-term movement of the index depends on the long-term links only, as the short-term links are successively replaced by their long-term counterparts.
For example, let the short- term indices for January to December 2001 be calculated as: I Dec 00:m01=P w00(Dec 00) (9.18) 00(Dec00) are the weights from 2000 price- where Wi updated to December 2000.

When weights for 2001 become available, this is replaced by the long-term link: I Dec 00:m01 i i i I Dec00:Dec01=P w01(Dec 00) I Dec 00:Dec 01 (9.19) i 01(Dec00) are the weights from 2001 price- where Wi backdated to December 2000.
The same set of weights from 2001 price-updated to December 2001 is used in the new short-term link for 2002: I Dec01:m02=P w01(Dec01) (9.20) : I Dec01:m02 i i 9.125 Using this method, the movement of the long- term index is determined by contemporaneous weights.

The method is conceptually attractive because the weights that are most relevant for most users are those based on consumption patterns at the time the price changes actually take place.
The method takes the process of chaining to its logical conclusion, at least assuming the indices are not chained more frequently than once a year.

As the method uses weights that are continually revised to ensure that they are representative of current consumer behaviour, the resulting index also largely avoids the substitution bias that occurs when the weights are based on the consumption patterns of some period in the past.
The method may therefore appeal to statistical oﬃces whose objective is to estimate a cost of living index.

9.126 Finally, it may be noted that the method involves some revision of the index ﬁrst published.

In some countries, there is opposition to revising a CPI once it has been ﬁrst published, although it is standard prac- tice for other economic statistics, including the national accounts, to be revised as more information and more up-to-date information become available.
This point is considered further below.

Decomposition of index changes 9.127 Users of the index are often interested in how much of the change in the overall index is attributable to the change in the price of some particular good or group of products, such as oil or food.

Alternatively, there may 171 CONSUMER PRICE INDEX MANUAL: THEORY AND PRACTICE be interest in what the index would be if housing or energy were left out.
Questions of this kind can be answered by decomposing the change in the overall index into its constituent parts.

9.128 Assume that the index is calculated as in equation (9.10) or (9.11).
The relative change of the index from tm to t can then be written as: I 0:t I 0:tm 1= P wb i I 0:tm i P wb I tm:t i i I 0:tm i 1 (9.21) Hence, a sub-index from tm to 0 enters the higher-level index with a weight of: i I 0:tm wb i P wb i I 0:tm i = i I 0:tm wb i I 0:tm (9.22) The eﬀect on the higher-level index of a change in a sub-index can then be calculated as: 1 Effect= I 0:t i I 0:tm i i I 0:tm wb i I 0:tm wb I 0:tm (I 0:t i With m=1, the formula (9.23) gives the eﬀect of a monthly change; with m=12, it gives the eﬀect of the change over the past 12 months.
i I 0:tm i (9.23) = ) 9.129 If the index is calculated as a chain index, as in equation (9.15), then a sub-index from tm enters the higher-level index with a weight of: i I k:tm w0 i I k:tm = I 0:k i (I 0:tm w0 i i (I 0:tmI 0:k) ) (9.24) The eﬀect on the higher-level index of a change in a sub- index can then be calculated as: Effect= = ) w0 I k:tm (I k:t i w0 i (I 0:tmI 0:k) i I k:tm i i I 0:tm I 0:t i I 0:k i (9.25) It is assumed that tm lies in the same link (i.e.
tm refers to a period later than k).

If the eﬀect of a sub- index on a higher-level index is to be calculated across a chain, the calculation needs to be carried out in two steps: one with the old series up to the link period, and one from the link period to period t. Table 9.8 Decomposition of index changes Weight Index 9.130 The calculation of the eﬀect of a change in a sub-index on a higher-level index is illustrated in Table 9.8.
The index is calculated in one link so that equation (9.25) may be applied for the decomposition.

For instance, the eﬀect in percentage points of the increase for housing from January 2002 to January 2003 can be calculated as 0.25/118.6 (120.0 110.0)=2.11 percen- tage points.

This means that, of the increase of 10.03 per cent in the all-items index, 2.11 percentage points can be attributed to the increase in the index for housing.
Some alternatives to ﬁxed weight indices 9.131 Monthly CPIs are, typically, arithmetic weighted averages of the price indices for the elementary aggregates, in which the weights are kept ﬁxed over a number of periods – which may range from 12 months to many years.

The repeated use of the same weights relat- ing to some past period b simpliﬁes calculation proce- dures and reduces data collection requirements.
It is also cheaper to keep using the results from an old expenditure survey than to conduct an expensive new one.

Moreover, when the weights are known in advance of the price collection, the index can be calculated immediately after the prices have been collected and processed.
9.132 The longer the same weights are used, how- ever, the less representative they become of current consumption patterns, especially in periods of rapid technical change when new kinds of goods and services are continually appearing on the market and old ones disappearing.

This may undermine the credibility of an index that purports to measure the rate of change in the total cost of a basket of goods and services typically consumed by households.
Such a basket needs to be representative not only of the households covered by the index, but also of expenditure patterns at the time the price changes occur.

9.133 Similarly, if the objective is to compile a cost of living index, the continuing use of the same ﬁxed basket is likely to become increasingly unsatisfactory the longer the same basket is used.

The longer the same basket is used, the greater the upward bias in the index is likely to become.
It is well known that the Laspeyres index has an upward bias compared with a cost of living index.

However, a Lowe index between periods 0 and t with weights from an earlier period b will tend to exceed the Laspeyres between 0 and t by an amount that increases the further back in time period b is (see Chapter 15).
2000 Jan. 02 Jan. Food 2 Clothing 3 Housing 4 Transport 5 Miscellaneous All items 172 Change in % from Jan. 02 to Jan. 03 Effect (contribution) % points of total change 2.53 1.26 2.11 0.84 3.29 10.03 % of total change 25.21 12.61 21.01 8.40 32.77 100.00 9.134 There are several possible ways of minimizing or avoiding the potential biases from the use of ﬁxed weight indices.
These are outlined below.

9.135 Annual chaining.

One way in which to mini- mize the potential biases from the use of ﬁxed-weight indices is obviously to keep the weights and the base period as up to date as possible by frequent rebasing and chaining.

Quite a number of countries have adopted this strategy and revise their weights annually.
In any case, as noted earlier, it would be impossible to deal with the changing universe of products without some chaining of the price series within the elementary aggregates, even if the weights attached to the elementary aggregates remain ﬁxed.

Annual chaining eliminates the need to choose a base period, as the weight reference period is always the previous year, or possibly the preceding year.

9.136 Annual chaining with current weights.


When the weights are changed annually, it is possible to replace the original weights based on the previous year, or years, by those of the current year, if the index is revised retro- spectively as soon as information on the current year’s expenditures becomes available.

The long-term move- ments in the CPI are then based on the revised series.

This is the method adopted by the Swedish Statistical Oﬃce, as explained above.
This method could provide unbiased results.

9.137 Other index formulae.

When the weights are revised less frequently, say every ﬁve years, another possibility would be to use a diﬀerent index formula for the higher-level indices instead of an arithmetic average of the elementary price indices.

One possibility would be a weighted geometric average.

This is not subject to the same potential upward bias as the arithmetic average.
More generally, a weighted version of the Lloyd–Moulton formula might be considered.

This for- mula takes account of the substitutions that consumers make in response to changes in relative prices, and should be less subject to bias for this reason.

It reduces to the geometric average when the elasticity of substitution is unity, on average.
It is unlikely that such a formula could replace the arithmetic average in the foreseeable future and gain general acceptance, if only because it cannot be interpreted as measuring changes in the value of a ﬁxed basket.

It could, however, be compiled on an experimental basis and might well provide a useful sup- plement to the main index.

It could at least ﬂag the extent to which the main index is liable to be biased and throw light on its properties.

9.138 Retrospective superlative indices.


Finally, it is possible to calculate a superlative index retrospectively.

Superlative indices, such as Fisher and To¨ rnqvist indices, treat both periods compared symmetrically and require expenditure data for both periods.

Although the CPI may have to be some kind of Lowe index when it is ﬁrst published, it may be possible to estimate a superlative index later when much more information becomes available about consumers’ expenditures period by pe- riod.
At least one oﬃce, the United States Bureau of Labor Statistics, publishes such an index.

The publica- tion of revised or supplementary CPIs raises matters of statistical policy, although users readily accept revi- sions in other ﬁelds of economic statistics.

Moreover, CALCULATING CONSUMER PRICE INDICES IN PRACTICE users are already confronted with more than one CPI in the European Union (EU) where the harmonized index for EU purposes may diﬀer from the national CPI.
Thus the publication of supplementary indices which throw light on the properties of the main index and which may be of considerable interest to some users seems justiﬁed and acceptable.

Data editing 9.139 This chapter has been concerned with the methods used by statistical oﬃces to calculate their CPIs.

This concluding section considers the data editing car- ried out by statistical oﬃces, a process that is very closely linked to the calculation of the price indices for the ele- mentary aggregates.
Data collection, recording and coding – the data capture processes – are dealt with in Chapters 5 to 7.

The next step in the production of price indices is data editing.

Data editing is here meant to comprise two steps: – detection of possible errors and outliers; – verifying and correction of data.

9.140 Logically, the purpose of detecting errors and outliers is to exclude errors or outliers from the index calculation.
Errors may be falsely reported prices, or they may be caused by recording or coding mistakes.

Also, missing prices because of non-response may be dealt with as errors.

Possible errors and outliers are usually iden- tiﬁed as observations that fall outside some pre-speciﬁed acceptance interval or are judged to be unrealistic by the analyst on some other ground.

It may also be the case, however, that even if an observation is not identiﬁed as a potential error, it may actually show up to be false.
Such observations are sometimes referred to as inliers.

Some- times, by chance, the sampling may have captured an exceptional price change, which falls outside the accep- tance interval but has been veriﬁed as correct.

In some discussions of survey data, any extreme value is des- cribed as an outlier.

The term is reserved here for extreme values that have been veriﬁed as being correct.

9.141 When a possible error has been identiﬁed, it needs to be veriﬁed whether it is in fact an error or not.
This clariﬁcation can usually be made by asking the respondent to verify the price, or by comparison with the price change of comparable items.

If the value is in fact an error, it needs to be corrected.

This can be done easily if the respondent can provide the correct price or, where this is not possible, by imputation or omitting the price from the index calculation.

If the value proves to be correct, it should be included in the index.
If it proves to be an outlier, it can be accepted or corrected according to a pre-deﬁned practice, e.g.

omitting or imputation.

9.142 Although the power of computers provides obvious beneﬁts, not all of these activities have to be computerized.

There should be a complete set of proce- dures and records that controls the processing of data, even though some or all of it may be undertaken without the use of computers.
It is not always necessary for all of one step to be completed before the next is started.

If the process uses spreadsheets, for example, with default imputations predeﬁned for any missing data, the index 173 CONSUMER PRICE INDEX MANUAL: THEORY AND PRACTICE can be estimated and re-estimated whenever a new observation is added or modiﬁed.
The ability to examine the impact of individual price observations on elemen- tary aggregate indices and the impact of elementary indices on various higher-level aggregates is a useful aid in all aspects of the computation and analytical processes.

9.143 It is neither necessary nor desirable to apply the same degree of scrutiny to all reported prices.

The price changes recorded by some respondents carry more weight than others, and statistical analysts should be aware of this.
For example, one elementary aggregate with a weight of 2 per cent, say, may contain 10 prices, while another elementary aggregate of equal weight may contain 100 prices.

Obviously, an error in a reported price will have a much smaller eﬀect in the latter, where it may be negligible, while in the former it may cause a signiﬁcant error in the elementary aggregate index and even inﬂuence higher-level indices.
9.144 There may be an interest in the individual ele- mentary indices, as well as in the aggregates built from them.

Since the sample sizes used at the elementary level may often be small, any price collected, and error in it, may have a signiﬁcant impact on the results for individual products or industries.

The veriﬁcation of reported data usually has to be done on an index-by-index basis, using the statistical analysts’ experience.
Analysts will also need the cooperation and support of the respondents to the survey to help explain unusual price movements.

9.145 Obviously, the design of the survey and ques- tionnaires also inﬂuences the occurrence of errors.

Hence, price reports and questionnaires should be as clear and unambiguous as possible to prevent misunderstandings and errors.

Whatever the design of the survey, it is important to verify that the data collected are those that were requested initially.
The survey questionnaire should prompt the respondent to indicate if the requested data could not be provided.

If, for example, a product is not produced any more and thus is not priced in the current month, a possible replacement would be requested along with details as to the extent of its comparability with the old one.

In the event that a respondent cannot supply a replacement, there are a number of procedures for deal- ing with missing data (also discussed in Chapter 7).
Identifying possible errors and outliers 9.146 One of the ways in which price surveys are diﬀerent from other economic surveys is that, although prices are recorded, the measurement concern is with price changes.

As the index calculations consist of com- paring the prices of matching observations from one period to another, editing checks should focus on the price changes calculated from pairs of observations, rather than on the reported prices themselves.
9.147 Identiﬁcation of unusual price changes can be accomplished by: – non-statistical checking of input data; – statistical checking of input data; – output checking.

These will be described in turn.

174 9.148 Non-statistical checking of input data.

Non- statistical checking can be undertaken by manually checking the input data, by inspection of the data pre- sented in comparable tables, or by setting ﬁlters.
9.149 When the price reports or questionnaires are received in the statistical oﬃce, the reported prices can be checked manually by comparing these to the previously reported prices of the same items or by comparing them to prices of similar items from other outlets.

While this procedure may detect obvious unusual price changes, it is far from certain that all possible errors will be detected.

It is also extremely time-consuming and, of course, it does not identify coding errors.
9.150 After the price data have been coded, the sta- tistical system can be programmed to present the data in a comparable tabular form.

For example, a table showing the percentage change for all reported prices from the previous to the current month may be produced and used for detection of possible errors.

Such tables may also include, for comparison, the percentage changes of pre- vious periods and 12-month changes.
Most computer programs and spreadsheets can easily sort the observa- tions according to, say, the size of the latest monthly rate of change, so that extreme values can easily be identiﬁed.

It is also possible to group the observations by elementary aggregates.

9.151 The advantage of grouping observations is that it highlights potential errors so that the analyst does not have to look through all observations.
A hierarchical strategy whereby all extreme price changes are ﬁrst identiﬁed and then examined in context may save time, though the price changes underlying elementary aggre- gate indices, which have relatively high weights, should also be examined in context.

9.152 Filtering is a method by which possible errors or outliers are identiﬁed according to whether the price changes fall outside some predeﬁned limits, such as plus or minus 20 per cent or even 50 per cent.
This test should capture any serious errors of data coding, as well as some of the cases where a respondent has erroneously reported on a diﬀerent product.

It is usually possible to identify these errors without reference to any other observations in the survey, so this check can be carried out at the data capture stage.

The advantage of ﬁltering is that it avoids the analyst having to look through a lot of individual observations.
The upper and lower limits may be set for the latest monthly change, or change over some other period.

Again, they should take account of the context of the price change, in that they may be speciﬁed by item or elementary aggregates or higher-level indices.

Larger changes for items with prices that are known to be volatile might be accepted without question.
For example, for monthly changes, limits of plus or minus 10 per cent might be set for oil prices, while for professional services the limits might be zero per cent to plus 5 per cent (as any price that falls is suspect), and for computers it might be 5 per cent to zero per cent (as any price that rises is suspect).

The limits can also be changed over time.


If it is known that oil prices are rising, the limits could be 10 per cent to 20 per cent, while if they are falling, the limits might be 10 per cent to 20 per cent.

The count of failures should be monitored regularly to examine the limits.

If too many observations are being identiﬁed for review, the limits will need to be adjusted, or the scope reﬁned.

9.153 The use of automatic deletion systems is not advised, however.
It is a well-recorded phenomenon in pricing that price changes for many products, especially durables, are not undertaken smoothly over time, but saved up to avoid what are termed ‘‘menu costs’’ asso- ciated with making a price change.

These relatively sub- stantial increases may take place at diﬀerent times for diﬀerent models of products and have the appearance of extreme, incorrect values.
To delete a price change for each model of the product as being ‘‘extreme’’ at the time it occurs is to ignore all price changes for the industry.

9.154 Statistical checking of input data.

Statistical checking of input data compares, for some time period, each price change with the change in prices in the same or a similar sample.

Two examples of such ﬁltering are given here, the ﬁrst based on non-parametric summary measures and the second on the log normal distribution of price changes.
9.155 The ﬁrst method involves tests based on the median and quartiles of price changes, so they are unaf- fected by the impact of any single ‘‘extreme’’ observa- tion.

Deﬁne the median, ﬁrst quartile and third quartile price ratios as RM, RQ1, and RQ3, respectively.

Then any observation with a price ratio that is more than a certain multiple C of the distance between the median and the quartile is identiﬁed as a potential error.

The basic approach assumes that price changes are normally dis- tributed.
Under this assumption, it is possible to estimate the proportion of price changes that are likely to fall outside given bounds expressed as multiples of C. Under a normal distribution, RQ1 and RQ3 are equidistant from RM.

Thus, if C is measured as RM (RQ1+RQ3)/2, then 50 per cent of observations would be expected to lie within plus or minus C from the median.

From the tables of the standardized normal distribution this is equivalent to about 0.7 times the standard deviation (s).
If, for example, C was set to 6, the distance implied is about 4s of the sample, so about 0.17 per cent of observations would be identiﬁed this way.

With C=4, the corre- sponding ﬁgures are 2.7s, or about 0.7 per cent of observations.

If C=3, the distance is 2.02s, so about 4 per cent of observations would be identiﬁed.

9.156 In practice, most prices may not change each month and the share of observations identiﬁed as possible errors as a percentage of all changes would be unduly high.
Some experimentation with alternative values of C for diﬀerent industries or sectors may be appropriate.

If this test is to be used to identify possible errors for further investigation, a relatively low value of C should be used.

9.157 To use this approach in practice, three mod- iﬁcations should be made: First, to make the calculation of the distance from the centre the same for extreme changes on the low side as well as on the high side, a transformation of the ratios should be made.
The transformed distance for the ratio of one price observation i, Si, should be: Si=1 RM=Ri if 0 < Ri < RM and Si=Ri=RM 1 if Ri RM: CALCULATING CONSUMER PRICE INDICES IN PRACTICE Second, if the price changes are grouped closely to- gether, the distances between the median and quartiles may be very small, so that many observations would be identiﬁed that had quite small price changes.
To avoid this, some minimum distance, say 5 per cent for monthly changes, should also be set.

 Third, with small samples the impact of one obser- vation on the distances between the median and quartiles may be too great.

Because sample sizes for some elementary indices are small, samples for similar elementary indices may need to be grouped together.

9.158 For a detailed presentation of this method, see Hidiroglou and Berthelot (1986).
The method can be expanded to also take into account the level of the prices.

Thus, for example, a price increase from 100 to 110 will be attributed a diﬀerent weight from the weight attrib- uted to a price increase from 10 to 11.

9.159 An alternative method can be used if it is thought that the price changes may be distributed log normally.
To apply this method, the standard deviation of the log of all price changes in the sample (excluding unchanged observations) is calculated and a goodness of ﬁt test (w2) is undertaken to identify whether the dis- tribution is log normal.

If the distribution satisﬁes the test, all price changes outside two times the exponential of the standard deviation are highlighted for further checking.
If the test rejects the log normal hypothesis, all the price changes outside three times the exponential of the standard deviation are highlighted.

The same caveats mentioned before about clustered changes and small samples apply.

9.160 The second example is based on the Tukey algorithm.

The set of price ratios is sorted and the highest and lowest 5 per cent ﬂagged for further atten- tion.
In addition, having excluded the top and bottom 5 per cent, exclude the price ratios that are equal to 1 (no change).

The arithmetic (trimmed) mean (AM) of the remaining price ratios is calculated.

This mean is used to separate the price ratios into two sets, an upper and a lower one.

The upper and lower ‘‘mid-means’’, that is, the means of each of these sets (AML, AMU), are then calculated.
Upper and lower Tukey limits (TL, TU) are then established as the mean plus (minus) 2.5 times the diﬀerence between the mean and the mid-means: TU=AM+2:5 (AMU AM) TL=AM 2:5 (AM AML) Then all those observations that fall above TU and below TL are ﬂagged for attention.

9.161 This is a simpler method similar to that based on the normal distribution.
Since it excludes all cases of no change from the calculation of the mean, it is unlikely to produce limits that are very close to the mean, so there is no need to set a minimum diﬀerence.

Its success will also depend on there being a large number of observa- tions on the set of changes being analysed.

Again, it will often be necessary to group observations from similar elementary indices.
For any of these algorithms, the comparisons can be made for any time periods, including the latest month’s changes or longer periods, in partic- ular, 12-month changes.
175 CONSUMER PRICE INDEX MANUAL: THEORY AND PRACTICE 9.162 The advantage of these two models of ﬁltering compared to the simple method of ﬁltering is that for each period the upper and lower limits are determined by the data and hence are allowed to vary over the year, given that the analyst has decided on the value of the parameters entering the models.
A disadvantage is that, unless the analyst is prepared to use approximations from earlier experience, all the data have to be collected before the ﬁltering can be undertaken.

Filters should be set tightly enough so that the percentage of potential errors that turn out to be real errors is high.

As with all automatic methods, the ﬂagging of an unusual observa- tion is for further investigation, as opposed to automatic deletion.

9.163 Checking by impact, or data output checking.


Filtering by impact, or output editing, is based on cal- culating the impact that an individual price change has on an index to which it contributes.

This index can be an elementary aggregate index, the total index, or some other aggregate index.

The impact that a price change has on an index is its percentage change times its eﬀec- tive weight.
In the absence of sample changes, the cal- culation is straightforward: it is the nominal (reference period) weight, multiplied by the price relative, and divided by the level of the index to which it is con- tributing.

So the impact on the index I of the change of the price of product i from time t to t+1 is ±wi ( pt+1 / pt)/It where wi is the nominal weight in the base period.
A minimum value for this impact can be set, so that all price changes that cause an impact greater than this change can be ﬂagged for review.

If index I is an ele- mentary index, then all elementary indices may be reviewed, but if I is an aggregative index, prices that change by a given percentage will be ﬂagged or not depending on how important the elementary index to which they contribute is in the aggregate.
9.164 At the lowest level, the appearance and dis- appearance of products in the sample cause the eﬀective weight of an individual price to change substantially.

The eﬀective weight is also aﬀected if a price observation is used as an imputation for other missing observations.

The evaluation of eﬀective weights in each period is possible, though complicated.

As an aid to highlighting potential errors, the nominal weights, as a percentage of their sum, will usually provide a reasonable approx- imation.
If the impact of 12-month changes is required to highlight potential errors, approximations are the only feasible ﬁlters to use, as the eﬀective weights will vary over the period.

9.165 One advantage of identifying potential errors in this way is that it focuses on the results.

Another advantage is that this form of ﬁltering also helps the analyst to describe the contributions to change in the price indices.
In fact, much of this kind of analysis is done after the indices have been calculated, as the analyst often wishes to highlight those indices that have con- tributed the most to overall index changes.

Sometimes the analysis results in a ﬁnding that particular industries have a relatively high contribution to the overall price change, and that is considered unrealistic.

The change is traced back to an error, but it may be late in the pro- duction cycle and jeopardize the schedule release date.
176 There is thus a case for identifying such unusual con- tributions as part of the data editing procedures.

The disadvantage of this method is that an elementary index’s change may be rejected at that stage.

It may be necessary to over-ride the calculated index, though this should be only a stopgap measure until the index sample is redesigned.

Verifying and correcting data 9.166 Some errors, such as data coding errors, can be identiﬁed and corrected easily.
Ideally, these errors are caught at the ﬁrst stage of checking, before they need to be viewed in the context of other price changes.

Dealing with other potential errors is more diﬃcult.

Many results that fail a data check may be judged by the analyst to be quite plausible, especially if the data checking limits are broad.

Some potential failures may only be resolved by checking the data with the respondent.
9.167 If a satisfactory explanation can be obtained from the respondent, the data can be veriﬁed or cor- rected.

If not, procedures may diﬀer.

Rules may be estab- lished that if a satisfactory explanation is not obtained, then the reported price is omitted from the index calcu- lation.

Alternatively, it may be left to the analyst to make the best judgement as to the price change.
If an analyst makes a correction to some reported data without veri- fying it with the respondent, the change may subse- quently cause problems with the respondent.

If the respondent is not told of the correction, the same error may persist in the future.

The correct action depends on a combination of conﬁdence in the analysts, the revision policy of the survey, and the degree of communication with respondents.

Most statistical organizations do not want to burden respondents unduly.
9.168 In many organizations, a disproportionate share of activity is devoted to identifying and following up potential errors.

If this practice leads to little change in the results, as a result of most reports ﬁnally being accepted, then the ‘‘bounds’’ on what are considered to be extreme values should be relaxed.
More errors are likely to be introduced by respondents failing to report changes that occur than from wrongly reporting changes, and the good will of respondents should not be unduly undermined.

9.169 Generally, the eﬀort spent on identifying potential errors should not be excessive.

Obvious mis- takes should be caught at the data capture stage.
The time spent in identifying observations to query, unless they are highly weighted and excessive, is often better spent treating those cases in the production cycle where things have changed – quality changes or unavailable prices – and reorganizing activities towards maintaining the relevance of the sample, and checking for errors of omission.
9.170 If the price observations are collected in a way that prompts the respondent with the previously repor- ted price, the respondent may report the same price as a matter of convenience.

This can happen even though the price may have changed, or even when the particular product being surveyed is no longer available.

As prices for many items do not change frequently, this kind of error is unlikely to be spotted by normal checks.
Often the situation comes to light when the contact at the responding outlet changes and the new contact has dif- ﬁculty in ﬁnding something that corresponds to the price previously reported.

It is advisable, therefore, to keep a record of the last time a particular respondent reported a price change.

When that time has become suspiciously long, the analyst should verify with the respondent that the price observation is still valid.
What constitutes too long will vary from product to product and the level of overall price inﬂation, but, in general, any price that has remained constant for more than a year is suspect.

9.171 Treatment of outliers.

Detection and treatment of outliers (extreme values that have been veriﬁed as being correct) is an insurance policy.
It is based on the fear that a particular data point collected is exceptional by chance, and that if there were a larger survey, or even a diﬀerent one, the results would be less extreme.

The treatment, therefore, is to reduce the impact of the exceptional observation, though not to ignore it as, after all, it did occur.

The methods to test for outliers are the same as those used to identify potential errors by sta- tistical ﬁltering, described above.
For example, upper and lower bounds of distances from the median price change are determined.

In this case, however, when observations are found outside those bounds, they may be changed to be at the bounds or imputed by the rate of change of a comparable set of prices.

This outlier adjustment is sometimes made automatically, on the grounds that the analyst by deﬁnition has no additional information on which to base a better estimate.
While such automatic adjustment methods are employed, this manual proposes caution in their use.

If an elementary aggregate is relatively highly weighted and has a rela- tively small sample, an adjustment may be made.

The general prescription should be to include veriﬁed prices; the exception should be to dampen them.

9.172 Treatment of missing price observations.

It is likely that not all the requested data will have been received by the time the index needs to be calculated.
It is generally the case that missing data turn out to be delayed.

Sometimes, the respondent may report that a price cannot be reported because neither the product, nor any similar substitute is being made any more.

Sometimes, of course, what started apparently as a late report becomes a permanent loss to the sample.

Diﬀer- ent actions need to be taken depending on whether the situation is temporary or permanent.
9.173 For the most appropriate strategy is to minimize the occurrence of missing observations.

Survey reports are likely to come in temporarily missing prices, CALCULATING CONSUMER PRICE INDICES IN PRACTICE over a period of time before the indices need to be cal- culated.

In many cases, they follow a steady routine; some respondents will tend to ﬁle quickly, others typically will be later in the processing cycle.

An analyst should become familiar with these patterns.
A computerized data capture system can ﬂag those reports that appear to be later than usual, well before the processing deadline.

Also, some data are more important than others.

Depending on the weighting system, some respondents may be particularly important, and important products should be ﬂagged as requiring particular scrutiny.
9.174 For those reports for which no estimate can be made, two basic alternatives are considered here (see Chapter 7 for a full range of approaches): imputation, preferably targeted, in which the missing price change is assumed to be the same as some other set of price changes; or an assumption of no change, as the preceding period’s price is used.
This latter procedure ignores the fact that some prices will prove to have changed, and if prices are generally moving in one direction, this will mean that the change in the index will be understated.

It is not advised.

However, if the index is periodically revised, this approach will lead to fewer subsequent revisions than imputations, since for most products, prices do not generally change in any given period.
Stand- ard imputation is to base the estimate of the missing price observation on the change of some similar group of observations.

9.175 There will be situations where the price is permanently missing because the product no longer exists.

As there is no replacement for the missing price, an imputation will have to be made for each period until either the sample is redesigned or until a replacement can be found.
It is, therefore, more important than in the case of temporarily missing reports, and requires closer attention.

9.176 The missing price can be imputed by using the change in the remaining price observations in the ele- mentary aggregate, which has the same eﬀect as remov- ing the missing observation from the sample, or by the change in a subset of other price observations for com- parable items.

The series should be ﬂagged as being based on imputed values.
9.177 Samples are designed on the basis that the products chosen for observation are representative of a wider range of products.

Imputations for permanently missing prices are indications of weakness in the sample, and their accumulation is a signal that the sample should be redesigned.

For indices where there are known to be a large number of disappearances in the sample, the need for replacements should be anticipated.
177 SOME SPECIAL CASES Introduction 10.1 This chapter focuses on a number of expendi- ture areas that pose particular problems for price index compilers, both in terms of identifying an agreed con- ceptual approach and also overcoming practical mea- surement diﬃculties.

Six areas have been selected for discussion, mainly from the service sector.

They are: – owner-occupied housing; – clothing; – telecommunication services; – ﬁnancial services; – real estate agency services; – property insurance services.
10.2 This chapter is therefore structured into six sections, in turn dealing with the problem areas listed above.

Under each section, any necessary theoretical considerations are discussed and relevant measurement issues explored.

Where appropriate, illustrative exam- ples of alternative approaches to the measurement of weights or price changes are provided, and the advan- tages and disadvantages are outlined.
10.3 It is important to note that the examples shown are neither deﬁnitive nor prescriptive, but rather provide broad guidance as to how the problem areas can be ap- proached.

User requirements, data availability and the statistical resources available are important factors that need to be taken into consideration in choosing an appropriate methodology.

Market conditions and pro- duct market regulations, which can diﬀer widely between countries, also have a critical impact on the choice of method.
Owner-occupied housing 10.4 The treatment of owner-occupied housing in consumer price indices (CPIs) is arguably the most dif- ﬁcult issue faced by CPI compilers.

Depending on the proportion of the reference population that are owner- occupiers, the alternative conceptual treatments can have a signiﬁcant impact on the CPI, aﬀecting both weights and, at least, short-term measures of price change.
10.5 Ideally, the approach chosen should align with the conceptual basis that best satisﬁes the principal purpose of the CPI.

However, the data requirements for some (or even all) of these options may be such that it is not feasible to adopt the preferred treatment.

Equally important, it may be diﬃcult to identify a single principal purpose for the CPI.
In particular, the dual use of CPIs as both macroeconomic indicators and also for indexa- 10 tion purposes can lead to clear tensions in designing an appropriate treatment for owner-occupied housing costs.

In these circumstances, it may be necessary to adopt a treatment that is not entirely consistent with the approach adopted for other items in the CPI.
In some countries, the diﬃculties in resolving such tensions have led to the omission of owner-occupied housing from the CPI altogether or the publication of more than one index.

10.6 The remainder of this section discusses the conceptual basis and data requirements for the use, payments and acquisitions approaches in turn.

Use 10.7 The general objective of this approach is to measure the change over time in the value of the ﬂow of shelter services consumed by owner-occupiers.
Detailed approaches fall under one of two broader headings: user cost or rental equivalence.

10.8 The user cost approach attempts to measure the changes in the cost to owner-occupiers of using the dwelling.

In the weighting base period, these costs comprise two elements: recurring actual costs, such as those for repairs and maintenance, and property taxes; and the opportunity cost of having money tied up in the dwelling rather than being used for some other purpose.
At its simplest, and where houses are purchased out- right, this latter element is represented by the rate of return available on alternative assets.

More usually, house purchase will be at least part ﬁnanced through mortgage borrowing.

In this case, opportunity cost can be viewed as an average of interest rates on mortgages and alternative assets, weighted by the proportion of the purchase price borrowed and paid outright, respectively.
10.9 Estimation of the base period weight for recur- ring actual costs such as expenditures on repairs and maintenance is relatively straightforward and generally obtainable from household expenditure surveys.

Simi- larly, the construction of price measures for these items presents few diﬃculties.

10.10 Estimation of for opportunity costs is more complicated and will require modelling.
One approach is to assume that all owner- occupiers purchased their dwellings outright at the beginning of the period and sold them at the end.

During the period their opportunity costs comprise the amount of interest forgone (i.e.

the amount of interest they might have earned by investing this money elsewhere) and depreciation.

Oﬀsetting these costs would be any capital gains earned on the sale of the dwellings.
Construction of the required measures of price change is likewise quite the base period weight 179 CONSUMER PRICE INDEX MANUAL: THEORY AND PRACTICE complicated (see Chapter 23 for a more complete dis- cussion) and, particularly for the depreciation element, a good deal of imputation is required.
Allowing for house purchases part ﬁnanced by mortgage borrowing, a typi- cal formula for user cost (UC) is: UC=rM+iE+D+RC K where M and E represent mortgage debt and equity in the home, and r and i represent mortgage interest rates and the rate of return available on alternative assets, respectively.

D is depreciation, RC other recurring costs and K capital gains.

10.11 No national statistical oﬃce is currently using the full user cost approach.
This partly reﬂects the conceptual and methodological complexity of the mea- sure, which may also make it diﬃcult to obtain wide- spread public support for the approach.

For this reason, the methodology is not discussed in detail here.

It is, however, worth noting that both the weights and the ongoing measures of price change are signiﬁcantly inﬂuenced by the relative rate of change in house prices.
Since the user cost formula is typically dominated by capital gains and interest rates, where house price inﬂation exceeds nominal interest rates the user cost weight is likely to be negative (implying a negative price for user cost).

10.12 In practice, it is possible to avoid some of these diﬃculties by adopting a variant or a narrower deﬁnition of user cost.
For example, some countries have adopted a variant of the user cost approach focusing on gross mortgage interest payments and depreciation, in part because these items are readily recognizable as key costs by home owners.

The former may be viewed as the cost of retaining housing shelter today, while the depreciation element represents current expenditure that would be required to oﬀset the deterioration and obsolescence in dwellings that would otherwise occur over time.
Meth- odologies for calculating actual average mortgage inter- est payments for index households are described in the section on the payments approach to owner-occupied housing costs, below.

10.13 Depreciation is a gradual process and so is best represented by the amount that needs to be put aside year by year as opposed to actual expenditures (which will typically be large but infrequent).
The base period weight for depreciation may be estimated from the current the owner-occupied housing stock market value of excluding land values, multiplied by an average rate of depreciation.

The latter may be derived from national accounts estimates of housing capital consumption.

Imputed this way, the appropriate price indicator should ideally be an index of house prices excluding land rather than an index of the costs of renovation work.
10.14 The rental equivalence approach attempts to measure the change in the price of the housing service consumed by owner-occupiers by estimating the market value of those services.

In other words, it is based on estimating how much owner-occupiers would have to pay to rent their dwelling.
Under this approach, it would be inappropriate also to include those input costs nor- mally borne by landlords such as dwelling insurance, major repair and maintenance, and property taxes as 180 this would involve an element of double counting.

The rental equivalence approach is recommended in SNA 1993 for measuring household consumption and is also used in constructing international comparisons of living standards.
10.15 Deriving the weight for rental equivalence requires estimating how much owner-occupiers would have paid in the weighting base period to rent their dwellings.

This is not something that owner-occupiers can normally be expected to estimate reliably in a house- hold expenditure survey.

In principle, however, it can be estimated by matching the dwellings of owner-occupiers with comparable dwellings that are being rented and applying those rents to the owner-occupied dwellings.
10.16 In practice, this raises a number of problems, particularly in countries where the overall size of the private rental market is small or if rented housing is of a diﬀerent type from owner-occupied housing in terms of general quality, age, size and location.

Direct imputa- tion from actual rents may also be inappropriate if the rental market is subject to price control.
In addition, owner-occupiers may be considered to derive signiﬁcant additional utility from features such as security of tenure and the ability to modify the dwelling, implying a need to make additional adjustments to the initial imputa- tions.

10.17 In those countries where the reference popu- lation for the CPI corresponds to all resident households, the estimation problem is identical to that faced by the national accountants and a collaborative approach would be beneﬁcial.
10.18 The corresponding price series for owner- occupiers’ rent can be derived from an actual rent index, except where such rents are subject to price control.

Depending on both the relative signiﬁcance of owner- occupiers to renters and the composition of the two markets in terms of dwelling characteristics, any existing rent surveys may need to be modiﬁed to meet the par- ticular requirements of an owners’ equivalent rent series.
If the total value of owners’ equivalent rent is signiﬁ- cantly larger than actual rents, the absolute size of the existing price sample may be deemed insuﬃcient.

If the characteristics of owner-occupied dwellings diﬀer sig- niﬁcantly from the overall rental market, the existing rent survey may also require stratifying more ﬁnely (e.g.

by type and size of dwelling, and by location).


The price measures for the diﬀerent strata can then be given dif- ferent weights when calculating the actual rents and the owners’ equivalent rent series, respectively.

10.19 While it may be acceptable to include sub- sidized and controlled prices in the actual rent series, these should not be used in calculating the owners’ equivalent rent series.
Given the increased signiﬁcance of rent prices in the overall index, it may also be necessary to pay greater attention to the measurement of price change for individual properties when tenancies change.

As this often presents landlords with an opportunity to refurbish properties and increase rents, the practice of regarding the whole of all such price changes as arising from quality change should be avoided.
Furthermore, the rent series may need to be quality-adjusted to take account of ongoing depreciation to housing structures.

This question is discussed in Chapter 23, paragraphs 23.69 to 23.78.

Payments 10.20 The item domain for a payments index is deﬁned by reference to actual outlays made by house- holds to gain access to consumer goods and services.
The set of outlays peculiar to owner-occupiers in the weighting base period includes: – down payments or deposits on newly purchased dwellings; – legal and real estate agency fees payable on property transfers; – repayments of mortgage principal; – mortgage interest payments; – alterations and additions to the dwelling; – insurance of the dwelling; – repair and maintenance of the dwelling; – property rates and taxes.
10.21 While it is conceivable to include all of these items in the index, it is generally agreed that at least some represent capital transactions that ought to be excluded from a CPI.

For example, while down payments and repayments of mortgage principal result in a running down of household cash reserves, they also result in the creation of a real asset (at least part of a dwelling) or in the reduction of a liability (the amount of mortgage debt outstanding).
Similarly, any cash expenditures on alter- ations and additions result in a running down of cash reserves oﬀset by increases in dwelling values.

In other words, those transactions which result in no net change to household balance sheets should be excluded.

10.22 The remaining items can be regarded as cur- rent expenditures which do not result in any oﬀsetting adjustments to household balance sheets.

It is therefore considered appropriate that these items be included in a payments-based CPI.
By deﬁning a payments index in this way, it is clear that the aggregate payments equal a household’s source of funds which comprise income after tax (wages, transfers, property income, insurance claims, etc.)

and net savings (as a balancing item).

It is for this reason that a payments-based CPI is commonly considered to be the best construct for assessing changes in net money incomes over time.
10.23 Estimation of gross expenditures on these items in the weighting base period is readily achievable via a household expenditure survey, as the items are generally reportable by households.

The construction of price indices for real estate agency fees and insurance is dis- cussed later in this chapter.

Indices for repair and main- tenance, and property rates and taxes are not considered particularly problematic so are not discussed here.
The remainder of this section is therefore devoted to the con- struction of price measures for mortgage interest charges.

10.24 The construction of price indices for mortgage interest charges is not altogether straightforward.

The degree of complexity will vary from country to country depending on the operation of domestic ﬁnancial mar- kets and the existence (or otherwise) of any income tax SOME SPECIAL CASES provisions applying to mortgage interest payments.
What follows therefore is a description of an overall objective and an illustrative methodology for producing the required index in the most straightforward of cases.

The methodology will require modifying to account for additional complexities that may be encountered in some countries.

10.25 The general approach may be summarized brieﬂy as follows.
Under a ﬁxed basket approach, the objective of the index is to measure the change over time in the interest that would be payable on a set of mort- gages equivalent to those existing in the weighting base period.

This base stock of mortgages will, of course, vary widely in age, from those taken up in the base period itself to those taken up many years previously.

In compiling a ﬁxed base index, the distribution of mort- gages by age is required to be held constant.
10.26 The amount of interest payable on a mortgage is determined by applying some rate of interest, ex- pressed as a percentage, to the monetary value of debt.

Changes in mortgage interest charges over time therefore can, in principle, be measured by periodically collecting information on a representative selection of mortgage interest rates, using these to derive an average interest rate, and then applying this to an appropriate debt ﬁg- ure.
At least for standard variable rate mortgages, interest due on the revalued stock of base period mort- gages may be derived simply with reference to current mortgage interest rates.

10.27 The main problem then is in determining the appropriate debt ﬁgure in each of the comparison peri- ods.
Since the real value of any monetary amount of debt varies over time according to changes in the pur- chasing power of money, it is not appropriate to use the actual base period monetary value of debt in calcula- tions for subsequent periods.

Rather, it is necessary ﬁrst to update that monetary value in each comparison period so that it remains constant in real terms (i.e.

so that the quantities underpinning the base period amount are held constant).
10.28 In order to do this, it is necessary to form at least a theoretical view of the quantities underpinning the amount of debt in the base period.

The amount of mortgage debt outstanding for a single household in the base period depends on the original house purchase price and loan-to-value ratio, and also the rate of repayment of principal since the house was purchased.
An equiva- lent value of debt can be calculated in subsequent com- parison periods by holding constant the age of the debt, the original value of the debt (as some ﬁxed proportion of the total value of the dwelling when the mortgage was initially entered into) and the rate of repayment of the principal (as some proportion of the original debt), and applying these factors to house prices for periods cor- responding to the age of the debt.
10.29 To illustrate, suppose a base period household purchased a dwelling ﬁve years earlier for $100,000 and ﬁnanced 50 per cent by mortgage.

If, between the time of purchase and the base period, the household repaid 20 per cent of this debt, then the outstanding debt on which base period interest charges were calculated would have been $40,000.
Now move to some subsequent 181 CONSUMER PRICE INDEX MANUAL: THEORY AND PRACTICE comparison period and suppose that it is known that house prices doubled between the period when the household originally purchased and the period ﬁve years prior to the comparison period.
The equivalent amount of outstanding debt in the comparison period would be calculated by ﬁrst taking 50 per cent of the revalued house price (of $200,000) to give $100,000, and then reducing this by the principal repayment rate (of 20 per cent) to give $80,000.
10.30 Under these assumptions, it is clear that the comparison period value of outstanding debt may be estimated directly from the base period value of out- standing debt solely on the basis of house price move- ments between ﬁve years prior to the base period and ﬁve years prior to the comparison period.
In other words, while preservation of original debt/equity ratios and rates of repayment of principal help in understanding the approach, estimates of these variables are not strictly required to calculate the required comparison period debt.

All that is required is the value of the outstanding debt in the base period, the age of that debt and a suit- able measure of changes in dwelling prices.
10.31 Now suppose that all mortgages are of the variable rate type, and that average nominal interest rates rose from 5 per cent in the base period to 7.5 per cent in the comparison period.

Interest payments in the two periods can be calculated as $2,000 and $6,000 respectively, and so the mortgage interest payments index for the comparison period is 300.0.

An identical re- sult may of course be found directly from index number series for debt and nominal interest rates.
The mortgage interest charges index equals the debt index multiplied by the nominal interest rate index divided by 100.

In this example, the debt index equals 200.0 and the nominal interest rate index equals 150.0.

Therefore the mortgage interest rate index equals (200.0150.0)/100 or 300.0.

This simple example also serves to illustrate the very important point that percentages (interest rates, taxes, etc.)

are not prices and cannot be used as if they were.


Percentages must be applied to some monetary value in order to determine a monetary price.

10.32 While the single-household example shown above is useful in explaining the basic concepts, it is necessary to devise a methodology that can be employed to calculate a mortgage interest charges index for the reference population as a whole.
The main complication when moving from the single-household to the many- household case is the fact that the age of the debt will vary across households.

Given the importance of re- valuing base period debt to maintain a constant age, this is no trivial matter.
While it is conceivable that infor- mation on the age of mortgage debt could be collected in household expenditure surveys, the additional respon- dent burden and the generally small number of house- holds reporting mortgages often serve to make estimates from this source unreliable.

Another option is to approach a sample of providers of mortgages (banks, building societies, etc.)

for an age proﬁle of their current mortgage portfolio.

This type of data is normally available and is generally reliable.
10.33 Table 10.1 illustrates how an aggregate debt price index can be constructed.

For the purpose of 182 illustrating the methodology, some simplifying assump- tions have been made: The index is assumed to be quarterly rather than monthly.

 The oldest age of mortgage debt is assumed to be between three and four years (in practice, it is nor- mally the case that debt older than eight years is insigniﬁcant).
Each annual cohort of debt is assumed to be dis- tributed evenly across the year.

 A quarterly index of dwelling prices (new and second- hand dwellings, including land) is available.

10.34 Column (1) of Table 10.1(a) contains index numbers for dwelling prices extending back four years prior to the base period for the debt series (quarter 1 of year 0).
Column (2) contains a four-quarter moving average of the ﬁrst series – this is required to reﬂect ‘‘yearly’’ prices to correspond with the debt cohorts, which are only available in yearly age groups in this example (if quarterly cohorts were available it would not be necessary to calculate the moving average series).

10.35 Columns (1) to (4) of Table 10.1(b) contain the calculated debt indices for each cohort re-referenced to Y0 Q1=100.
These series are simple transformations of the series in column (2) of Table 10.1(a), each with a diﬀerent starting point.

For example, the debt series for that cohort contracted for between three and four years ago has as a starting point the index number from Y4 Q4 (i.e.

113.9) in column (2), and the series for debt aged between two and three years starts from Y3 Q4 (i.e.

118.7) and so on.
118.7) and so on.

Column (5) of Table 10.1(b) contains the aggregate debt index which is derived by weight- ing together the indices for the four age cohorts.

The weights are derived from data from ﬁnancial institutions on debt outstanding by age, revalued to period Y0 Q1 prices.
10.36 A nominal mortgage interest rate index num- ber series is obtained by calculating average quarterly interest rates on variable rate mortgages from a sample of lending institutions (starting in period Y0 Q1) and presenting them in index number form.

The nominal interest rate series can then be combined with the debt series to calculate the ﬁnal mortgage interest rate char- ges series, as illustrated in Table 10.2.
10.37 The construction of equivalent indices for ﬁxed interest mortgages is more complicated in so far as an interest charges index has to be calculated separately for each age cohort of debt to reﬂect the fact that interest payable today, on a loan four years old, depends on the interest rate prevailing four years ago.

This requires the compilation of a nominal ﬁxed interest rate index extending back as far as the dwelling price series.
To the extent that the interest rates charged on ﬁxed interest loans also depend on the duration of the loan, calcula- tion of the nominal ﬁxed interest rate series is also more complex.

The additional complexity of these indices may make the construction of a mortgage interest charges index impractical for countries where ﬁxed interest rate mortgages predominate.
10.38 The construction of the index for mortgage interest payments is predicated on the assumption that Table 10.1 Calculation of a mortgage debt series (a) Dwelling price index Year Y4 Y3 Y2 Y1 Y0 Quarter Q1 Q2 Q3 Q4 Q1 Q2 Q3 Q4 Q1 Q2 Q3 Q4 Q1 Q2 Q3 Q4 Q1 Q2 Q3 Q4 SOME SPECIAL CASES Original house price index (1) Four-quarter moving average of (1) (2) (b) Debt index Year Quarter Age of debt Y0 Q1 Q2 Q3 Q4 3–4 years Wt = 10% (1) 100.0 101.2 102.5 103.4 2–3 years Wt = 20% (2) 100.0 100.6 100.9 101.3 1–2 years Wt = 30% (3) 100.0 100.7 101.6 102.2 0–1 year Wt = 40% (4) 100.0 100.7 101.1 101.7 Weighted average (5) 100.0 100.7 101.4 101.9 the purpose of the mortgage is to ﬁnance the purchase of the dwelling (hence revaluation of debt by changes in dwelling prices).
However, it is increasingly common, particularly in developed countries, for households to draw down on the equity they have in their home.

That is, households may take new or additional mortgages, or redraw part of the principal already paid to ﬁnance other activities, for example to purchase a large con- sumer durable such as a car or a boat, to go on holiday or even to purchase stocks and bonds.
If these alter- native uses of the funds made available by way of it may be appropriate to mortgages are signiﬁcant, regard at least some proportion of mortgage interest charges as the cost of a general ﬁnancial service rather Table 10.2 Calculation of a mortgage interest charges series Year Quarter Y0 Q1 Q2 Q3 Q4 Debt index (1) 100.0 100.7 101.4 101.9 Nominal interest rates index (2) 100.0 98.5 100.8 101.5 Mortgage interest charges index (1) (2)/100 (3) 100.0 99.2 102.2 103.4 than a housing cost.
For that proportion of the debt deemed to be used for other purposes, it would be more appropriate to use a general index of price inﬂation for debt revaluation purposes.

Acquisitions 10.39 The item domain for an acquisitions index is deﬁned as all those consumer goods and services acquired by households.
Those countries which compile their CPIs on an acquisitions basis have generally con- cluded that the principal purpose of their CPI is to provide a measure of price inﬂation for the household sector as a whole.

Based on the view that price inﬂation is a phenomenon peculiar to the operation of markets, the domain is also normally restricted to those consumer goods and services acquired in monetary transactions.
That is, consumer goods and services provided at no cost to households by governments and non-proﬁt institu- tions serving households are excluded.

10.40 The expenditures of owner-occupiers that could be included in an acquisitions index are: – net purchases of dwellings (i.e.
purchases less sales by the reference population); – direct construction of new dwellings; – alterations and additions to existing dwellings; 183 CONSUMER PRICE INDEX MANUAL: THEORY AND PRACTICE – legal and real estate agency fees payable on property transfers; – repair and maintenance of dwellings; – insurance of dwellings; – property rates and taxes.

10.41 The construction of price indices for real estate agency fees and insurance is discussed later in this chapter.
Indices for repair and maintenance, and prop- erty rates and taxes are not considered particularly problematic so are not discussed here.

The remainder of this section is therefore devoted to a discussion of the issues involved in constructing measures for dwelling purchase, construction, and alterations and additions.
An advantage of the acquisitions approach is that, con- sistent with the treatment of most other goods and ser- vices in the CPI, the owner-occupied housing index will reﬂect the full price paid for housing.

Moreover, it is not aﬀected by methods of ﬁnancing for house purchase.

10.42 As CPIs are constructed to measure price change for a group of households in aggregate (the reference or target population), the index should not include any transactions that take place between those households.
In the case of an index covering all private households, the weight should only reﬂect net additions to the household sector owner-occupied housing stock.

In practice, net additions will mainly comprise those dwellings purchased from businesses (newly constructed dwellings, company houses, or rental dwellings) and those purchased from or transferred from the government sector plus any purchases, for owner-occupation, of rental dwellings from reference population households.
If the CPI is constructed for some subgroup of the population (e.g.

wage and salary earners), the weight should also include purchases from other household types.

10.43 Economists regard all housing as ﬁxed capital and hence would exclude purchases of dwellings from household consumption.

While this is unambiguously the case for housing purchased for rental, the case is less clear-cut when it comes to housing for owner-occupa- tion.
Although households recognize the likelihood of making capital gains when they purchase housing and invariably regard their dwelling as an asset, they also commonly cite the primary motivation for the purchase of a dwelling as being to gain access to a service (i.e.

shelter and security of tenure).


From the households’ perspective, therefore, the costs borne by owner-occu- piers in respect of their principal dwelling represent a mix of investment and consumption expenditure, and the total exclusion of these costs from an acquisitions-based CPI can lead to a loss of conﬁdence in the CPI by the population at large.
Particularly in those countries where rental sectors are relatively small, with limited opportu- nities for substitution between owner-occupation and renting, it might be argued that the consumption element dominates.

10.44 The problem confronting compilers of CPIs is how to separate the two elements so as to include only the consumption element in the CPI.
Although there is no single agreed technique, one approach is to regard the cost of the land as representing the investment 184 element and the cost of the structure as representing the consumption element.

The rationale for this is that while the structure may deteriorate over time and hence be ‘‘consumed’’, the land remains at constant quality for all time (except under extremely unusual circum- stances).
As the land (or location element) accounts for most of the variation in observable prices for otherwise identical dwellings sold at the same point in time, the exclusion of land values may also be seen as an attempt to exclude asset price inﬂation from the CPI.

(Measures of asset price inﬂation are, of course, useful in their own right.)
10.45 Derivation of weighting base period expendi- tures on the net acquisition of dwellings (excluding land), the construction of new dwellings, and alterations and additions to existing dwellings poses some pro- blems.
Although household expenditure surveys may yield reliable estimates of the amounts households spend on alterations and additions, and construction of dwellings, it is unlikely that they will provide reliable estimates of net expenditures on existing dwellings exclusive of the value of the land.

10.46 An alternative approach is to combine data from censuses of population and housing and building activity surveys.
Population censuses normally collect information on housing tenure, from which average annual growth in the number of owner-occupier house- holds represents a good proxy for net additions to the housing stock.

Building activity surveys are also con- ducted in most countries, providing data on the total value of dwellings constructed.

These data can be used to estimate the average value of new dwellings, which can then be applied to the estimated volumes derived from the population census.
Of course, the suitability of this approach would need to be assessed by each country and may be complicated if the CPI relates only to some sub- set of the total population.

10.47 The price index is required to measure the change over time in existing dwelling structures, newly constructed dwellings, and alterations and additions.
As the appropriate price for existing dwelling structures is current replacement cost, an index measuring changes in prices of newly constructed dwellings is also appropriate for this purpose.

Given that the prices for both newly constructed dwellings and alterations and additions are, in principle, determined by costs of building materials, labour costs and producers’ proﬁts, it may also be sat- isfactory to construct a single price sample for all ele- ments.
The requirement for a separate price sample for alterations and additions will depend on the relative signiﬁcance of this activity and whether the material and labour components diﬀer signiﬁcantly from those for a complete dwelling (e.g.

if alterations and additions are predominantly to kitchens and bathrooms).

In all cases, it is important that the price indices are mix-adjusted to eliminate price variations that reﬂect changes in the characteristics of newly constructed dwellings.
10.48 The type of dwelling constructed in individual countries will signiﬁcantly inﬂuence the complexity and cost of constructing appropriate price measures.

If each newly constructed dwelling is essentially unique (i.e.

designed to meet site or other requirements) it will be necessary to adopt ‘‘model pricing’’.
This requires selec- tion of a sample of building ﬁrms, identifying samples of recently constructed dwellings and collecting prices for constructing identical dwellings in subsequent periods (exclusive of site preparation costs, which will vary from site to site).

This approach is likely to entail signiﬁcant costs for the respondents.

Moreover, care needs to be taken to ensure that the supplied prices truly reﬂect all prevailing market conditions.
That is, prices need to reﬂect the amount builders could realistically expect to be able to charge in the current market rather than the prices they would like to be able to charge based on conditions prevailing in some prior period.

10.49 In a number of countries, a signiﬁcant pro- portion of newly constructed dwellings are of the type referred to as ‘‘project homes’’.

These are homes that builders construct on a regular basis from a suite of standard designs maintained for this purpose.
This practice is most feasible in countries where a signiﬁcant proportion of new dwelling construction takes place in new developments (i.e.

land recently developed or re- developed speciﬁcally for residential housing).

Where project home construction is signiﬁcant in scale, then it is possible to select a sample of these project homes for pricing over time, safe in the knowledge that the prices provided will be actual transaction prices (again, priced net of any site preparation costs).
Even if project homes do not account for the majority of new dwellings con- structed, they may still provide a representative measure of overall price change.

10.50 In pricing project homes, it is necessary to monitor the selected sample to ensure that the selected plans remain representative and to detect changes in quality arising from modiﬁcations in design and changes to basic inclusions.

Whenever a change is made to the plans, the change in overall quality has to be estimated.
For physically measurable characteristics, such as a small increase in the overall size of the dwelling, it may be assumed that the change in quality is proportional to the change in the relevant quantity.

Other changes, such as the addition of insulation, inclusion of a free driveway and so on, will need to be valued, preferably in terms of current value to the consumer.
These could be estimated by obtaining information on the amounts that con- sumers would have to pay if they were to have the items provided separately (the option cost method).

An alter- native is to ask the builder if a cash rebate is available in lieu of the additional features.
Where plans are modiﬁed to meet changed legal requirements, the consumer has no choice in purchase and so it is acceptable to classify the full change in price as pure price movement (even though there may be some discernible change in quality).

Clothing SOME SPECIAL CASES are only available in particular seasons and, unlike sea- sonal fruit and vegetables, the speciﬁc items on sale in one season (say summer) may not return the following year.
In addition to seasonal availability, the physical characteristics of some items of clothing can also change as a result of changing fashions.

10.52 The remainder of this section seeks to provide a general description of the clothing market applicable to most countries, discusses the most signiﬁcant pro- blems faced by index compilers and looks at some op- tions for overcoming or at least minimizing these.
The clothing market 10.53 Most countries experience at least some cli- matic variation throughout the year.

The number of discrete ‘‘seasons’’ may range from two (‘‘wet’’ and ‘‘dry’’, summer and winter) up to the four experienced in most regions (winter, spring, summer and autumn).

Items of clothing tend to fall into two categories: those that are available in one season only, and those that are available all year round.

10.54 Clothing (whether seasonal or not) is also subject to changes in fashion.
The fashion for trousers can change from straight legged to ﬂared; jackets from single-breasted to double-breasted; shirts from button- down collar to not; skirts from long length to short length, and so on.

10.55 Even within categories of garments which are not unduly aﬀected by seasonal inﬂuences or general changes in fashions, the garments that are available for pricing from one period to the next can vary greatly.
Retailers change suppliers in order to seek the best prices or to maintain an image of a constantly changing range in order to attract shoppers.

Many producers will also frequently change product lines in order to maintain buyer appeal.

The practice of single producers using diﬀerent and changing brands as a marketing tool is also common.
Isolated countries that rely predominantly on imported clothing also face the additional problem of discontinuities in supply because of shipping failures or even the whim of importers.

10.56 The often short life cycles of speciﬁc items, and whole categories of items in the case of seasonal items, mean that retailers have to pay particular atten- tion to inventory control, since they cannot aﬀord to be left with large volumes of stock that they cannot sell.
This is most commonly handled by progressively dis- counting or marking down prices throughout the esti- mated life cycle of an item.

10.57 The fragmented and changing nature of the clothing market invariably means that price index com- pilers have to strike a balance between the ideal require- ments for index purposes and the cost of data collection (of both prices and characteristics that may be required to make quality adjustments).
10.51 Clothing is a semi-durable good and its treat- ment is not aﬀected by the conceptual basis chosen for the CPI (acquisitions, use or payments).

Particular fea- tures of the clothing market do, however, create pro- blems for price index compilers.

Although clothing is purchased throughout the year, many types of clothing Approaches to constructing indices for non-seasonal clothing 10.58 Even where seasonality is not a problem, the construction of a price index for clothing is not a simple task.
The range of available items can diﬀer signiﬁcantly 185 CONSUMER PRICE INDEX MANUAL: THEORY AND PRACTICE across outlets, making central determination and detailed speciﬁcation of items to be priced ineﬀective.

The brands and styles of particular garment types can also vary signiﬁcantly over time in individual outlets, requiring close attention to procedures for replacing items and making quality adjustments.
10.59 Although it is virtually impossible to set out speciﬁc procedures that will be applicable in all coun- tries, it is possible to develop a set of guidelines to help avoid the most signiﬁcant pitfalls.

In developing these guidelines, the key objective is to maximize the number of usable price quotations (for a given collection cost) in any month, and to minimize the incidence of measures of price change being aﬀected by changes in quality.
10.60 In some circumstances, it may be possible to identify ‘‘national’’ speciﬁcations to be priced at each outlet (e.g.

brand X, model Y jeans).

The use of these types of speciﬁcations can help minimize the eﬀort that needs to be put into quality adjustment, and movements in prices of these items can provide a useful benchmark against which to assess the movements of other items.
Reliable identiﬁcation of such items necessitates ongoing relationships with the buyers for large chains, or large domestic producers or importers.

These sources need to be contacted on a regular basis to identify the current range of items, the extent of their availability across the country and any planned changes (including changes in style and quality as well as deletions from and additions to the range).
This information may be used proactively to update speciﬁcations or descriptions of items to be priced in the ﬁeld, so minimizing the incidence of price collectors attempting to price items that are no longer available.

It can also be used to assist in the quantiﬁca- tion of any quality changes.

10.61 For some items where availability by brand varies, it may be possible to identify a number of brands which are assessed as being of equal quality (e.g.

dif- ferent brands of T-shirts).


In these cases, price collectors could be provided with the list of equivalent brands and instructed to price the cheapest one of these available at each outlet without having to ensure that the same brand is priced this time as on the last visit.
The argu- ment for this practice is that, if the brands are truly equivalent, discerning shoppers will purchase the cheapest at the time of purchase, and to reﬂect this in the CPI will result in an index that more closely follows the experience of households.

Clearly, the success or otherwise of this technique depends vitally on the assessment of the ‘‘equality’’ of brands which, while largely a matter of judgement, may be assisted by an analysis of past price behaviour.
In general, brand equality might be indicated by narrow longer-term price dispersion and a tendency for brands to swap prices over time or outlets.

10.62 In other cases it might be appropriate to restrict sampled items to a subset of brands without regarding the brands as equivalent.

For example, a number of brands of jeans might together dominate the market but with the availability of the individual brands varying by outlet.
In these cases, price collectors could be provided with a list of acceptable brands and instructed to price the most representative of these brands at each 186 outlet.
Once the initial selection has been made, price collectors should be instructed to record the speciﬁc brand and model priced at each outlet, and should continue to price that speciﬁcation on subsequent visits until such time as it ceases to be stocked (or it becomes clear that it is no longer representative of the sales of that particular outlet).

10.63 The clothing market has become so diverse that it is not always possible to specify centrally either the item to be priced or even the brand (or brands).
In these cases, it is necessary to give price collectors much greater discretion when it comes to selecting the indivi- dual items for pricing.

To avoid the selection of inap- propriate items, it is important for price collectors to be provided with guidelines to assist in this process.
At the very least, they should be instructed to select the brand and model that the retailer advises is both representative and is expected to be stocked for some time (little advantage is to be gained from selecting an item which, while popular, has been purchased by the retailer on a one-oﬀ basis and is thus unlikely to be available for pricing in subsequent periods).

10.64 More sophisticated guidelines can incorporate a checklist of features that the selected item should match as closely as possible.
These features should be ranked from most to least important, and it should be clear which features the selected item possesses and which it does not (either from the detailed description recorded by the price collector or through the comple- tion of a separate feature pro-forma).

In addition to brand (or acceptable brands), where possible, the list might include features such as: – fabric type (e.g.

cotton, wool, linen); – weight of the fabric (e.g.


heavy, medium, light); – existence of a lining; – number of buttons; – type of stitching (e.g.

single, double).

10.65 It is recognized that high fashion items pose particular diﬃculties in terms of quality adjustment.

There is certainly clear potential for such items to bias the CPI towards the end of their life cycle when prices may be heavily discounted and sales volumes are low.
For example, compilers need to guard against the dan- ger that items leave the index at a heavily discounted price to be replaced by items that are on sale at the full price (which for a highly fashionable item may be at a premium).

More generally, any decision on the inclusion of high fashion items ought certainly to reﬂect the intended reference population of the index, for example where this excludes households at the upper end of the income distribution.
Replacement of items and quality change 10.66 Even for garment types that are available all year round, there remains a strong need to replace items or to otherwise recognize changes in item characteristics.

It is therefore important to ensure that procedures are established to minimize any bias resulting from changes in the quality of items priced.

10.67 The appropriate conceptual basis for assessing changes in the quality of garments is from the perspective of value to the consumer.
In other words, a garment can be said to be of diﬀerent quality to another garment if it is valued diﬀerently by the consumer.

The diﬃculty con- fronted by index compilers is that quality diﬀerences are only observable in terms of changes in the physical characteristics of garments (including brand), some of which will have an impact on customer value and some of which will not.

The problem is how to distinguish between them.


10.68 To assist in this task it is important to develop guidelines for selecting replacement items, with the general objective of minimizing the quality diﬀerence between the old and new items.
For most items, research has shown that brand is an important price- and quality-determining characteristic (particularly for items that have a signiﬁcant fashion element) and so, in the ﬁrst instance, an eﬀort should be made to select a replacement from the same brand (but noting the danger that as brands go out of fashion they become less representative).
As this will not always be possible, it is useful to enlist experts in the trade to assist in drawing up a list that classiﬁes brands into quality groups along the following lines: – exclusive brands, usually international brands, mostly sold in exclusive stores; – higher-quality brands, well-known brands at the national level (which may also include international brands); – average quality brands; – other or unknown brands.
10.69 If it is not possible to select a replacement from the same brand, the fallback should be to select a replacement from a brand in the same quality group.

Similarity of price should never be the guiding objective when a substitute variety has to be chosen.

10.70 Once a replacement item has been selected, a detailed description of the new item needs to be recorded.
The physical diﬀerences between the old and new items should be described in as much detail as possible to enable the index compiler to assess whether the replacement item is comparable (i.e.

of equal quality) to the old item or not.


As a general guide, changes such as single rows of stitching replacing double rows, of lighter-weight fabrics replacing heavier-weight ones, reductions in the number of buttons on shirts, reductions in the length of shirt tails, disappearance of linings and so on should be regarded as changes in quality.

Changes in physical characteristics attributable solely to changes in fashion (e.g.
straight leg to ﬂared leg trousers) should not be regarded as quality changes.

10.71 Where an item is assessed as not being com- parable, action will need to be taken to remove the impact of the quality change from the index.

There are a number of approaches that may be taken to value the quality diﬀerence: Industry experts may be asked to place a cash value on the diﬀerences.
The statistical oﬃce may arrange for some index compilers to receive additional training to become SOME SPECIAL CASES commodity experts able to estimate the value of such changes themselves.

 Hedonic methods may be employed if resources per- mit.

Descriptions of hedonic techniques for clothing can be found in Liegey (1992) and Norberg (1999).
10.72 Each of these methods requires that the changes in the quality-determining characteristics (such as quality of material and standard of manufacture) are quantiﬁ- able.

If such information is not available, implicit quality adjustment methods may have to be used.

In this case, it is important that the price for the outgoing speciﬁcation is returned to a normal price before it is removed from index calculation.
Approaches to including seasonal clothing in the consumer price index 10.73 The practices adopted by statistical agencies for handling seasonal clothing in CPIs vary widely, ranging from complete exclusion of such items to var- ious methods of imputation of prices of items that are unavailable at a particular time of year, or to systems of weights that vary throughout the year.
In some respects, the treatment of seasonal clothing raises similar issues to those found in dealing with fashion items, in particular reﬂecting the short life cycles of products and the like- lihood of price-discounting during those cycles.

10.74 This section describes some practical alter- natives for indices constructed using the traditional annual basket approach to produce a monthly CPI (i.e.
systems of explicitly changing weights are not explored, nor is the use of year-on-year changes as proposed in Chapter 22).

Further, the examples will be restricted to the so-called multiple basket approach because of the inherent diﬃculty of making quality adjustments be- tween seasons in the so-called single basket approach.
(The single basket approach takes the view that, say, summer and winter seasonal items are diﬀerent varieties of the same article, whereas the multiple basket approach takes the view that they are completely diﬀerent articles.)

10.75 CPI compilers may choose to exclude seasonal clothing from the CPI altogether.

While this might sim- plify the job of compiling the index, it clearly reduces the representativeness of the basket.
This might be con- sidered as the option of last resort and will cause pre- sentational diﬃculties from the point of view of external users, particularly where relative expenditure on seasonal clothing is high.

Including seasonal items makes the basket more representative of consumption patterns but complicates the process of compiling the index.

In reach- ing a decision, it will be necessary to strike a balance between representativeness and complexity (cost).
Where seasonal items are excluded, their expenditure weight should be distributed among non-seasonal counterparts.

10.76 Six possible approaches to constructing aggre- gate clothing price indices in the presence of seasonal items are described below.

A synthetic set of prices is used (see Table 10.3) to illustrate the various options.
For simplicity, it is assumed that there are only three categories of clothing: those available all year (non- seasonal); and two seasonal categories (labelled summer and winter here).
The two seasons are assumed to be 187 CONSUMER PRICE INDEX MANUAL: THEORY AND PRACTICE Table 10.3 Synthetic price data to illustrate approaches to constructing clothing price indices Month Year Y 1 Non- seasonal Summer seasonal Winter seasonal Year Y Non- seasonal 113 114 115 116 117 118 120 121 122 123 124 126 Summer seasonal Winter seasonal Year Y + 1 Non- seasonal Summer seasonal Winter seasonal 110 90 70 110 90 70 125 100 80 125 100 80 127 128 130 131 132 133 135 136 137 139 140 142 non-overlapping and the prices of the seasonal varieties are contrived to show progressive discounting over the course of each season.
The prices of the non-seasonal items show a steady rate of growth.

Within each cate- gory, prices are assumed to be for items of identical physical characteristics (or alternatively, to have been adjusted to remove the eﬀects of changes in physical characteristics).

10.77 The price indices have been compiled with a base period of month 1 in year 0 and extend for 24 months (prices are provided for year Y1 in order to impute base period prices for the winter seasonal item).
For the purpose of weighting, it is assumed that each of the seasonal categories accounts for 25 per cent of expenditure, while non-seasonal items account for the remaining 50 per cent.

For ease of computation, imputation is based on the simple arithmetic average of the price movements of the available series (including movements from imputed to real prices), though in practice these imputations would be based on weighted averages.
Tables 10.4 to 10.6 present the calculated indices and monthly percentage changes for summer seasonal, winter seasonal and total clothing, respectively, based on the alternative methodologies described below.

10.78 Exclude seasonal items.

This is the simplest option from an index construction point of view, but suﬀers from a lack of representativeness, which may be a cause of concern to some users.

In this example, only 50 per cent of expenditures would be directly represented in the index.
Clearly, the greater the relative expenditure on seasonal items, the more users are likely to be con- cerned about the lack of representativeness of the index.

The results for this index are shown in column (1) of Table 10.6 and may be used as a benchmark against which the following options can be assessed.

10.79 Impute only on items available all year.

This approach is one of the targeted imputation approaches.
In this case, the out of season prices for both summer and winter items are imputed based only on the move- ment in the prices of those items available all year round.

The results for the summer and winter items are shown in column (1) of Tables 10.4 and 10.5, respec- tively, while the total clothing index is shown in column (2) of Table 10.6.

188 10.80 Impute on all available items.

This approach imputes all missing prices based on the movements in all available prices of related or similar items.
This approach is similar in principle to the approach that would be taken in the case of a missing price observa- tion.

Prices for seasonal items are collected while they are observable, and when out of season are imputed based on items available all year round together with other seasonal items if available.

The results are shown in column (2) in Tables 10.4 and 10.5, and in column (3) of Table 10.6.

10.81 Carry forward of last observed price.


This simpler variant of the methods described above involves the carry forward of the last observed prices for sea- sonal items during the months when such prices are unavailable.

This approach would not normally be recom- mended in the general case where prices are not available for non-seasonal items, on the grounds that the likely downward bias imparted could easily be avoided by observing the price of some similar item that is available.
But where a whole class of goods is unavailable and hence unobservable, and particularly where price move- ments are not strongly correlated with other items, carry forward of prices may be seen as an acceptable approach.

The results are shown in column (3) in Tables 10.4 and 10.5, and in column (4) of Table 10.6. it is preferable to determine in advance during which months seasonal prices will be collected.
This helps prevent distortion of the index through collection of possibly atypical prices for seasonal items unexpectedly available outside those periods when they would normally be available.

Such decisions should be subject to regular review on the basis of market developments.

this approach, 10.82 Under 10.83 Return to normal, then impute.

This approach requires the index compiler to estimate the ‘‘normal’’ price for the item during the ﬁrst month when it is unavailable (out of season).
This estimated normal price is then imputed forward until such time as the item becomes available again.

Compared to the methods discussed so far, this approach is designed to avoid artiﬁcial depression of the aggregate index beyond the end of season, following progressive discounts over the item’s short life cycle.
Table 10.4 Alternative price indices for summer seasonal clothing Table 10.5 Alternative price indices for winter seasonal clothing Month Impute only on items available all year (1) Impute on all available items Carry forward of last observed price Return to normal, then impute Include ﬁrst seasonal observation, then impute Month (2) (3) (4) (5) Impute only on items available all year (1) Impute on all available items Carry forward of last observed price Return to normal, then impute Include ﬁrst seasonal observation, then impute (2) (3) (4) (5) SOME SPECIAL CASES Index numbers Monthly percentage changes 18.2
100.0
Index numbers Monthly percentage changes 17
22 23 24 10.84 There are some problems with this procedure.
Particularly during periods of high inﬂation, it will be diﬃcult to determine what the normal price is.

More generally, it can be argued that the procedure reduces the objectivity of the index.

In the illustrative examples presented here, the normal price to which the item is returned is the price observed at the start of the season.
Compared with the previous three approaches, it can be seen that this has the eﬀect of shifting the price increase from the commencement of the next season to imme- diately after the current season, i.e.

the index records a sharp price change when none is observable.

The results are shown in column (4) in Tables 10.4 and 10.5, and in column (5) of Table 10.6.

10.85 Include only the ﬁrst seasonal observation, then impute.
This approach requires that seasonal items be priced only once per season, when they ﬁrst appear in the marketplace.

This ﬁrst observed price is then imputed forward until the item is priced again at the commence- ment of the next season.
The rationale for this technique 189 CONSUMER PRICE INDEX MANUAL: THEORY AND PRACTICE Table 10.6 Alternative price indices for total clothing Month Only items available all year round (1) Impute only on items available all year (2) Impute on all available items (3) 100.0 93.8 87.2 88.0 88.7 89.5 100.5 94.1 87.6 88.3 89.0 90.5 110.1 102.4 96.6 97.4 98.1 98.9 115.0 106.8 100.1 101.6 102.3 103.8 Carry forward of last observed price (4) Index numbers 100.0 95.9 91.8 92.2 92.7 93.1 114.8 106.9 99.1 99.5 99.9 100.8 113.8 108.5 104.9 105.3 105.8 106.2 130.0 120.0 112.1 113.0 113.5 114.3 Monthly percentage changes 22.4
0.9 12.8 4.6 3.4 0.4 0.4 0.4 22.4 7.7 6.6 0.8 0.4 0.8 Return to normal, then impute (5) Include ﬁrst seasonal observation, then imput (6) 100.0 100.9 101.8 102.7 103.5 104.4 105.6 106.5 107.4 108.2 109.1 110.9 112.5 113.4 115.2 116.1 117.0 117.9
is that it is a means of adjusting for the quality degra- dation of seasonal items associated with the commonly observed feature of falling prices throughout the season.
Further, if it is desirable that the index behave as if it were constructed as a moving year index (see Chapter 22), then this approach provides a cost-eﬀective alter- native that also accommodates changing seasons (e.g.

when the items that were in season last March do not appear until April this year).
10.86 On the downside, in fully discounting obser- vable price movements through a seasonal item’s life cycle, an implicit assumption is made that all such movements reﬂect quality changes with no change in underlying price.

This is not likely to fully accord with user perceptions of price evolution and, unless similar techniques are employed for fashion items, it can be argued that the approach is inconsistent.

The results are shown in column (5) in Tables 10.4 and 10.5, and col- umn (6) in Table 10.6.
Summary comments 10.87 First, it is worth noting that the consequences of imputing price changes for baskets of seasonal items based 190 on the price movements for other items of clothing is equivalent to allocating the weight for seasonal items to other items when they are out of season, so avoiding the complexity involved in systems of explicitly changing weights.
In these circumstances, some care needs to be taken in the presentation of estimates of the contribution of both seasonal and non-seasonal items to the change in the aggregate CPI.

The standard practice of determining an item’s contribution to the total change in the CPI is to multiply the item’s previous period (price-updated) weight by its percentage change.

Only those seasonal items for which prices are actually measured in the current period will contribute to the change in the aggregate index.
Similarly, though only non-seasonal items will contribute to the change in the aggregate index when seasonal items are out of season, the standard measure of their con- tribution will be understated.

This is mainly an issue of presentation, although some compilers might prefer to present assessments of contributions only down to the level that includes both the seasonal and non-seasonal baskets.
10.88 There is likely to be a range of views across countries, and indeed users, concerning the appropriate treatment of seasonal items within a CPI.

There is likely to be a particular diversity of views about whether the quality of seasonal items should be regarded as dimin- ishing over the life of the season or not and, if so, whether a similar approach should (or can) be taken in respect of fashion items.
The example data set was contrived so that each category displayed broadly constant growth in prices on a year-on-year basis.

Those users primarily interested in measures that best capture persistent or underlying price pressures in the economy are likely to prefer those approaches which do not yield signiﬁcant variations in the rate of price change that are solely attributable to how the statistical agency treats seasonal items.
Such users may prefer that seasonal items be excluded al- together or that only the ﬁrst seasonal observation be included with prices for other months being imputed.

10.89 What is clear is that national statistical oﬃces need to carefully weigh up user requirements, theoretical issues, costs and the implications of alternative ap- proaches before settling on the methodology to be adopted.

Telecommunication services 10.90 The global telecommunications sector has undergone rapid change in recent years.
Technological innovation has resulted in a proliferation of new services while deregulation has led to sharp growth in the number of providers in many countries.

Taken together, these factors have resulted in suppliers adopting a range of new strategies to diﬀerentiate their services in order to attract and retain customers.
10.91 Characteristics of particular signiﬁcance to compilers of price indices are: fewer linear pricing schedules and the adoption of diﬀerent pricing structures across providers; the increasing tendency to oﬀer contracts that bundle services together in diﬀerent ways to appeal to dif- ferent types of consumers; SOME SPECIAL CASES rapid changes in the contracts oﬀered to consumers as an eﬀective means of encouraging the take-up of the ever-increasing range of services.
10.92 Increasingly, telecommunication companies oﬀer services via plans that require customers to enter into longer-term contractual arrangements with the providers.

This also poses problems for index compila- tion.

Two broad types of plan are typically oﬀered.

The ﬁrst has no ﬁxed duration and makes allowance for the provider to change pricing structures with advance notice to the consumer.
The second and increasingly more popular type provides a ﬁxed term contract (gen- erally of one to two years) with prices ﬁxed for the duration of the contract.

These plans are diﬀerentiated by charging diﬀerent prices for diﬀerent services.

For example, a simple plan may be diﬀerentiated by charging more for monthly line rental but less for local calls, so appealing to users who make a higher volume of local calls.
The emergence of new tailored plans designed to maximize customer demand overall is continuous.

10.93 If statistical agencies follow traditional sam- pling approaches and select price schedules according to some base period set of plans, and follow them until they expire, no price changes will be observed (likewise if plans expire and replacements are linked to show no change).
The marketplace reality, by contrast, is that unit values for telecommunication services have been declining signiﬁcantly in many countries.

10.94 All struggling to statistical agencies are develop methodologies capable of coping with the com- plexities of this sector.

In particular, it is recognized that current best practice approaches have diﬃculty in ac- counting for substitutions across providers and in ade- quately accounting for changes in the quality of the services provided.
10.95 With the telecommunications sector under continual change, statistical practices need to be kept under constant review.

Statistical agencies that are con- sidering the construction of telecommunications indices for the ﬁrst time, or considering reviewing their current practices, are advised to seek out the most recent re- search in this ﬁeld.
Nevertheless, this section seeks to provide a general description of four approaches that are currently used by national statistical agencies to measure changes in the prices of telecommunication services.

The approaches, in increasing order of cost, are: – representative items – matched samples; – representative items – unit values; – customer proﬁles; – sample of bills.

10.96 Each approach is brieﬂy described and poten- tial deﬁciencies noted.
There is no ﬁrm recommendation on the best approach as the choice will depend largely on the market conditions prevailing in individual countries, the sophistication of the index compilation system in use, and the extent of access to accurate and timely tele- communication services data.

Depending on these fac- tors, it may be appropriate to use diﬀerent approaches for diﬀerent telecommunication services, or even for the diﬀerent services of speciﬁc providers.
191 CONSUMER PRICE INDEX MANUAL: THEORY AND PRACTICE Representative items – matched samples 10.97 This approach mirrors traditional techniques adopted elsewhere in the CPI.

Total expenditure of reference group households on telecommunication ser- vices in the weighting base period is derived from sources such as household expenditure surveys.
A sample of service providers is approached to obtain information on revenue by types of services (such as line rental, local calls, international calls, handset sales or rentals, con- nection fees, voicemail services, Internet charges and so on) and a number of these are selected as representative items with weights derived from the revenue data.
10.98 For each representative item, a sample of detailed speciﬁcations (such as a telephone call from location A to location B, at time X, of duration Y min- utes) is drawn up suﬃcient to represent the range of speciﬁc services purchased by consumers within each representative item.

This sample of speciﬁcations is held constant from period to period, and movements in the indices for representative items are computed, based on the movements in the prices of this matched sample of the approach.
speciﬁcations.

Table 10.7 illustrates 10.99 The list of representative items (the lowest level in the structure) generally does not need to cover all telecommunication services, but those selected should be suﬃcient to be representative of price behaviour as a whole, in particular taking account of published tariﬀs.

Expenditures on those services not selected for pricing should be distributed over the other services within that general class for the purpose of deriving weights.
For example, the expenditures on any ﬁxed line services not selected for pricing should be distributed over those ﬁxed line services selected.

10.100 Compared to suppliers of goods, service providers have an almost inﬁnite capacity to tailor both the services and the prices they charge, for example based on the time at which the service is provided.
A telephone call of ﬁve minutes’ duration at 8 a.m. can be regarded as a diﬀerent product to an equivalent call made at 8 p.m., Table 10.7 An illustrative index structure for telecommuni- cation services (representative item approach) Fixed line services Telephone connection costs Telephone line rental Local calls Long-distance national calls International calls Mobile telephones Connection costs Handset purchase or rental National calls International calls Payphones Local calls Internet services Connection fees Usage fees 192 and service providers are able to charge diﬀerent prices for these calls.
Representative items therefore need to be described in suﬃcient detail to capture all the price- determining characteristics.

10.101 Furthermore, given the ease with which providers can adjust the diﬀerential aspects of their pricing schedules (such as the time span designated as peak and the duration of a call before a diﬀerent rate applies), it is necessary to use a suﬃcient number of varied speciﬁcations to capture these aspects reliably.
It is not suﬃcient to simply describe a call as peak or oﬀ-peak, or from zone 1 to zone 2.

Illustrative examples of the types of speciﬁcations that may be applicable for two representative items – international calls (ﬁxed line) and usage fees (Internet services) – are provided in Table 10.8.

10.102 It is assumed that the origin of both the tele- phone calls and Internet access is also identiﬁed.

All times are domestic.


It should also be noted that the nature of Internet access generally precludes pricing on the basis of access, and hence the timing of access can- not be as tightly deﬁned as for international telephone calls; instead, all speciﬁcations are for total monthly use.
10.103 The most costly aspect of this approach therefore is obtaining the data required to establish the representative items and to identify suitable speciﬁca- tions, as this will require detailed information from service providers.

Once implemented, most price infor- mation should be readily available from published fee schedules, so minimizing the burden on respondents between reviews of the speciﬁcations.
10.104 The dynamic nature of the telecommunica- tion sector and the common use of the pricing mecha- nism to change consumer behaviour are likely to require that the speciﬁcations be updated relatively frequently.

When a speciﬁcation disappears (i.e.

a particular plan is no longer oﬀered), all eﬀorts must be made to ﬁnd a suitable comparison speciﬁcation.
Where speciﬁcations are replaced, it is possible to argue that because diﬀerent plans involve diﬀerent conditions of sale they are fun- damentally diﬀerent products.
It is equally reasonable Table 10.8 Examples of speciﬁcations of telecommunica- tion services Representative item Examples of speciﬁcations International calls (ﬁxed line) Usage fees (Internet) Plan A: Call to Athens at 8 a.m. on a Friday, duration 10 minutes Plan B: Call to London at 9 p.m. on a Saturday, duration 5 minutes Plan A: Call to New York at 11 a.m. on a Wednesday, duration 20 minutes Plan B: Call to Paris at 7 p.m. on a Sunday, duration 15 minutes Plan A: Call to Durban at 8 p.m. on a Monday, duration 30 minutes Plan A: 10 hours dial-up connect time between 4 p.m. and 7 p.m. weekends, total download 20 Mb Plan B: 20 hours dial-up connect time between 6 p.m. and midnight week- days, total download 50 Mb Plan C: Permanent broadband con- nection, total download 100 Mb to question whether all of the price diﬀerence between plans is due to quality diﬀerences, particularly in light of the evidence of ever-increasing volumes and reductions in unit values.
The diﬃculty lies in quantifying the quality diﬀerences.

Although hedonic techniques oﬀer some prospects for resolving this dilemma, they are costly to implement.

Representative items – unit values 10.105 The unit value approach is similar to the previous approach, with the exception that speciﬁcations are not priced.

The price for each representative item is calculated from revenue and quantity data collected from the service provider.
For example, the price for national long-distance calls can be derived as the total revenue received from such calls divided by the number of call-minutes.

Similarly, in the case of monthly line rental fees, the price can be calculated as the total rev- enue from line rental divided by the total number of subscribers.

10.106 Compared to the matched sample approach, the unit value approach attributes all of the diﬀerence between plans, and time and duration of calls to price (i.e.
the quality diﬀerence is assumed to be zero).

The unit value approach is also seen as providing a method for accounting for price change when the items are subject to a proliferation of discount schemes or promotions (e.g.

$2 to call anywhere for as long as you like for the next week).
While the approach avoids some of the customer sampling choices inherent in other methodologies, com- pilation does rest on analysis of aggregate company data and so is likely to be less timely than methodologies based on pre-published prices.

Moreover, care needs to be exercised with this approach to ensure that the mea- sure is not aﬀected by undesirable compositional changes (see Chapter 9, where unit value indices are discussed in more detail).
A unit value index should only be con- structed for truly homogeneous items.

This points to a requirement for deﬁning the representative items at a relatively ﬁne level of disaggregation.

For example, international calls may need to be further subdivided by destination to avoid changes in unit values arising purely from shifts in the numbers of calls made to diﬀerent destinations.
10.107 Although this approach appears to address at least some of the known deﬁciencies of the matched sample approach, it is likely to have a medium- to long- term downward bias and, unless implemented carefully, it is likely to exhibit period-to-period volatility because of compositional shifts, if only as a result of seasonal variations in usage patterns.

There are also a number of respondent and data quality aspects that need to be considered.
The unit value approach imposes a greater data burden on service providers, who often regard revenue and quantity data as highly commercially sen- sitive.

To be eﬀective, the service providers also need to be able to furnish data relating only to households (i.e.

they have to be able to separate out revenue and quan- tities relating to businesses) and the revenue information needs to conform to the requirements of the index.
For example, some service providers may record certain SOME SPECIAL CASES discounts as a marketing expense, rather than a reduc- tion in revenue as is required for the unit value index.

Customer proﬁles 10.108 For marketing purposes, telecommunication companies often classify their customers according to their volume of service use.

Although the number of categories can vary, a common approach is to use a three- way classiﬁcation: low-volume, medium-volume and high-volume customers.
Service providers analyse cus- tomer usage patterns by category when developing new plans targeted speciﬁcally at each group.

National reg- ulatory authorities may also be in a position to provide detailed customer use proﬁles on a conﬁdential basis.

10.109 Statistical agencies can take a similar ap- proach for the construction of price indices by devising proﬁles which reﬂect the average usage patterns for each category of consumer.
Costs faced by these average consumers in each period can then be estimated by reference to the rates set out in that plan that is currently most commonly applicable to each customer category.

Variations on this general theme include estimation of costs based on the plan that would deliver the cheapest overall cost to the consumer (based on the simplifying assumption of cost-minimizing consumer behaviour with perfect knowledge).
This has the advantage of providing a clear basis for choosing a comparable replacement should an existing package cease to be available.

Alter- natively, costs to each customer group may be estimated with reference to several plans, where sales information indicates that this is a closer approximation to reality.

The overall index is derived by weighting together the results from these user proﬁles according to information about the relative importance of each category of con- sumer.
10.110 In constructing the aggregate index, these calculations are likely to be made for a representative sample of service providers, exploiting information on their overall market share for sampling or weighting purposes if available.

This opens up the possibility of fully exploiting all the possible relevant permutations of proﬁles and companies.

Information on the distribu- tion of customer proﬁles by service provider may, how- ever, not be available or at least very costly to obtain.
Table 10.9 gives an example of a proﬁle for mobile tele- phone services, taken from Beuerlein (2001), which describes the current approach used in the German CPI.

10.111 Consistent with the ﬁxed basket approach, the activity of consumers (in terms of numbers and types of calls) is held constant between comparison periods.

Prices may, of course, change when not ﬁxed by contract or when plans are replaced.
Index compilers may also allow rates to change in response to a changing mix of plans within customer categories.

This approach assumes that plan changes, as such, fundamentally represent price change rather than quality change, but it eliminates the cruder compositional eﬀects associated with the unit value approach, which does not take account of cus- tomer proﬁles.
10.112 The success of this approach is determined by the degree to which the proﬁles truly reﬂect consumer 193 CONSUMER PRICE INDEX MANUAL: THEORY AND PRACTICE Table 10.9 Example of a user proﬁle for mobile phone services Speciﬁcation Unit Rare callers Low-volume callers Average callers Total length of calls Length of individual call Minutes 16 Type A Type B Calls1 Within the same network Number Beyond the network Seconds 35 Seconds 65 Number 20 8 Number 12 42 45 95 36 12 24 96 45 115 72 24 48 1The calls are distributed over times of the day and days of the week so that it is possible to take account of changes in the delimitation of between peak and oﬀ-peak, weekday and weekend tariﬀs.
Source: Beuerlein (2001).

behaviour and therefore a great deal of thought needs to be put into their development.

The construction of the customer proﬁles will require a high degree of co- operation from service providers and, given the known volume changes, they will require updating at reason- ably regular intervals, possibly more frequently than other items in the CPI basket.
Data on plan usage by customer category for each index compilation period (month or quarter) may also be required if compilers decide to allow for such eﬀects.

Sample of bills 10.113 This method can be seen as a more reﬁned application of the customer proﬁle approach.

A ﬁxed level of service activity from an actual sample of cus- tomers is priced each month rather than deﬁning proﬁles representative of the average monthly activities of cus- tomers.
A sample of customers should be selected from each category of customer (low-, medium- and high- volume customers) and, ideally, the bills (or activity statements) should cover a full year’s activity.

10.114 The advantages of this approach compared to the customer proﬁle approach are: It is able to take account of any within-year variations in customer behaviour (e.g.

a higher incidence of international calls associated with religious or cultural events of signiﬁcance).
It better reﬂects the diversity of consumer behaviour by identifying actual activities (i.e.

calls actually made by a sample of consumers).

 It accommodates within each bill any instances of annual charges.

 It allows for the detection and recording of other sources of price change associated with customers’ overall relationship with the service provider (e.g.
where overall discounts are provided when aggregate monthly spending exceeds certain values, or where an aggregate discount is provided if customers acquire bundles of services from a single provider, such as ﬁxed line phone plus Internet).

the 10.115 Calculation of requires monthly information on the relative signiﬁcance of var- ious plans by customer category (which can then be randomly allocated across the sampled bills).
With the index still 194 bill sample repriced each period, the resulting index measures the cost of a full year’s consumption at the prices prevailing in each index period compared to the same cost at base prices.

This assumes that the quality diﬀerence between old and new plans is zero for house- holds’ changing plans.
Because of the generally larger number of bills (compared with the number of available proﬁles), price changes can be reﬂected more gradually, as the proportion of bills priced using each plan can better mirror the changing population distribution.

10.116 As with the proﬁle approach, it is important that the sample of bills is updated regularly to reﬂect changes in consumption patterns and the take-up of new services such as call-waiting, voicemail and text mes- saging.
Although, with adequate sampling, the bill ap- proach is likely to provide a better measure of the aggregate rate of price change for telecommunication services as a whole, it may not be best suited to the cal- culation of separate indices for the components of those services (depending on whether overall or bottom-line discounts are oﬀered).

The approach is also data intensive, requiring a large number of calculations each period and thus a sophisticated data processing system.
Financial services 10.117 The construction of reliable, comprehensive price indices for ﬁnancial services in CPIs is in its infancy.

Given the increasing use of ﬁnancial services by house- holds, however, national statistical agencies are coming under pressure to account for at least some ﬁnancial services in their CPIs.

There is a particularly strong demand for CPIs to include those fees and charges faced by households in respect of deposit and loan accounts held with ﬁnancial institutions.
10.118 The construction of price indices for ﬁnancial services is inherently diﬃcult, as there is no unanimous view about which ﬁnancial services ought to be included in the CPI, or indeed about precisely how they should be measured.

The discussion in this section attempts to present what might be regarded as the majority view based on what is practically feasible.

Much of the ma- is based on Fixler and Zieshang (2001), Frost terial (2001) and Woolford (2001).
10.119 Common examples of ﬁnancial services acquired by households include ﬁnancial advice, cur- rency exchange, services associated with deposit and loan facilities, services provided by fund managers, life insur- ance oﬃces and superannuation funds, stockbroking services, and real estate agency services.
The range of items explicitly regarded as ﬁnancial services for inclu- sion in a CPI, and also the way in which they are mea- sured, will depend on the principal purpose of the CPI and hence on whether an acquisitions, use or payments approach is employed.

10.120 Where a payments approach is used, the gross interest payable on mortgages is often included as a cost of owner-occupied housing (see paragraphs 10.4 to 10.50 above).
In the interests of strict consistency, this might imply that the CPI should also include consumer credit charges (measured in a similar way to mortgage interest charges), as well as gross outlays on direct fees and charges paid in respect of other ﬁnancial services.
In practice, and as noted in the earlier section on housing costs, the treatment of housing sometimes diﬀers in concept from other interest charges in national CPIs, partly reﬂecting mixed objectives for the overall index combined with public perceptions of the importance of this item within overall budgets.

The speciﬁc require- ments for a payments approach will not be discussed further here as the principles are either described else- where (e.g.
under owner-occupied housing) or are rela- tively straightforward.

10.121 Assuming that households acquire all of their ﬁnancial services from the private sector (i.e.

they are not generally subsidized by governments or provided by non- proﬁt institutions serving households), the acquisitions and use approaches take an identical view of the measurement of ﬁnancial services.
In terms of coverage, however, some proponents of the use approach take a more restrictive view of which services should be included by limiting the scope to only those ﬁnancial services which are acquired to directly facilitate current household consumption.

10.122 Under the more restrictive view of coverage, it is argued that the use of some ﬁnancial services is inex- tricably linked with capital (or investment) activity.
This suggests that such activities should be considered outside the scope of CPIs intended to provide measures of chan- ges in consumption prices.

Proponents of this view often draw upon national accounts practices as the starting point.

For example, SNA 1993 classiﬁes expenses asso- ciated with the transfer of real estate (real estate agents’ commissions, legal fees, and government taxes and char- ges) as part of gross ﬁxed capital formation.
It is impor- tant to note, however, that the CPI is not constrained to follow the practices adopted for national accounting.

Rather, individual countries will need to make decisions on the item coverage of the CPI which best meets the domestic requirements of the price index itself.
10.123 One broad deﬁnition that could be adopted for the coverage of ﬁnancial services within the CPI is: all those services acquired by households in relation to the acquisition, holding and disposal of ﬁnancial and real assets, including advisory services, except those acquired for business purposes.

This deﬁnition serves two purposes.

First, it distinguishes between the services facilitating the transfer and holding of assets and the assets themselves.
Second, it makes no distinction between whether the underlying asset is a real asset or a ﬁnancial asset.

10.124 The degree of complexity involved in placing a value on ﬁnancial services acquired by households and constructing the companion price indices varies mark- edly by service.

Three speciﬁc examples reﬂecting current Australian research are used to illustrate the issues: currency exchange, stockbroking, and deposit and loan facilities.
Real estate agency services are discussed sepa- rately in this chapter (see paragraphs 10.149 to 10.155) because they may be classiﬁed as either a housing expense or a ﬁnancial service.

Currency exchange 10.125 For weighting purposes, the estimation of the base period expenditures incurred by households SOME SPECIAL CASES in exchanging domestic currency for currencies of other countries is, in principle, relatively straightforward and should be reportable in household expenditure surveys.
10.126 Construction of the companion price index is more complex.

The service for which a price is required is that of facilitating the exchange of domestic currency for that of another country (the acquisition of an asset – foreign currency).

The price for the service is usually speciﬁed in terms of some percentage of the domestic currency value of the transaction.
These percentage margins may change only rarely, with service providers relying on the nominal value of the transactions increasing over time to deliver increases in fee receipts.

The price required for index construction purposes is the monetary value of the margin (i.e.

the amount determined by applying the percentage rate to the value of the currency transaction).

To measure price change over time, the index compiler has to form a the quantity underpinning the original view about transaction.
10.127 The purchase of foreign currency can be seen as facilitating the purchase of some desired quantity of foreign goods and services (e.g.

expenditure on foreign travel, or direct import of a commodity).

The service price in comparison periods would be expressed as the amount payable on the conversion of a sum of domestic currency corresponding to that sum of foreign currency required to purchase the same quantum of foreign goods and services purchased in the base period.
10.128 A practical translation implies that the ori- ginal foreign currency amount is indexed forward using changes in foreign prices, and then converted to domestic currency at the prevailing exchange rate, with the pre- vailing percentage margin applied to this new amount to deliver the current price.

This current price would be compared to the base price to derive the measure of price change.
Although the ideal measure for indexing forward the foreign currency amount would be an index speciﬁcally targeting those foreign goods and services purchased by resident households, this is unlikely to be feasible.

A practical alternative is to use the published aggregate CPI for the foreign countries.

10.129 If a single margin (percentage rate) does not apply to all transactions (e.g.
diﬀerent rates apply to diﬀerent size transactions), then the price measure should be constructed by reference to a representative sample of base period transactions.

The value margin for each transaction in the current period in the domestic currency would be determined by the current domestic currency value of each transaction and the current period percentage margin applying to each.
This captures any price change resulting from the value of an underlying transaction moving from one price band to another.

Stockbroking services 10.130 Consider the case of the purchase of a parcel of shares in a publicly listed company.

In most countries, the purchase has to be arranged through a licensed broker (stockbroker).
The total amount paid by the purchaser generally comprises three elements: an amount 195 CONSUMER PRICE INDEX MANUAL: THEORY AND PRACTICE for the shares (the asset); a fee for the brokerage service; and some form of transaction tax (stamp duty).

10.131 The tax should be considered part of the cost of acquiring the shares, as opposed to being part of the price of the security.

The tax should be included along with the brokerage cost in the CPI.
This is consistent with both the intention of the tax and the more com- monly accepted basis for the valuation of the shares.

(It also proves convenient to adopt this principle here, as it allows for the – perhaps less contentious – comparable treatment of taxes on banking services.)

Allowing for current tax schedules poses no diﬃculty in that they will be widely available in all countries.
10.132 Working from the premise that stockbrokers’ fees are more likely to follow a step function than a linear function, a price measure would be constructed as fol- lows.

First, select a representative sample of transactions (domestic currency values) and calculate the tax payable and the fees payable by reference to the respective sched- ules.
The taxes and fees payable in subsequent periods are calculated by ﬁrst indexing forward the values of the sample transactions and then applying current fee and tax schedules to the revalued transactions.

This methodology raises two main issues.

First, what is the most appropriate index for revaluing the transactions and, second, how should the current schedule of fees be determined?

10.133 The quantum underlying share transactions can be regarded as forgone consumption, i.e.
the quan- tity of goods and services that could have been purchased instead.

The value of a constant quantum of consump- tion forgone in successive comparison periods therefore will vary with consumer prices.

In this case, the obvious choice for an escalator would be the CPI itself, based on current period preliminary estimates, or the previous period’s result.
However, the use of a single period’s movement in the CPI (either previous or current) has the potential to result in the prices of stockbroking services moving in a way that is unlikely to reﬂect reality.

This would be particularly evident where, for example, the current or previous period’s CPI was inﬂuenced sig- niﬁcantly by some one-oﬀ, temporary or unusual price change (e.g.

an oil price shock, or change to health care arrangements).
Any ‘‘echoing’’ of abnormal shorter- term price changes through the precise treatment of stockbrokers’ or similar fees is likely to stretch public credibility in the CPI.

As an alternative, a 12-month moving average CPI might be employed, itself consistent with a base period comprising a full year’s activity.

10.134 Alternatively, it might be argued that the quantum of shares could be revalued in subsequent periods in line with movements in equity prices them- selves.
According to this view, the price of equities may be seen as an important inﬂuence on the actual costs of storing forgone consumption in much the same way as tax and fee schedules speciﬁc to equity purchases are allowed to enter the calculations described above.

The strong argument against this treatment is that it assumes that households have a desire to own equities per se, rather than using them simply as an appropriate vehicle to store forgone consumption.
Moreover, the introduc- tion of equity prices within the price indicator is likely to impart additional short-term volatility to the CPI.

196 10.136 10.135 Competition in the stockbroking industry means that there is unlikely to be a common fee schedule.

If individual brokers adhere reasonably closely to an in- house fee schedule, obtaining copies of these schedules should be a relatively simple matter.
On the other hand, if no such fee schedules exist, then a survey of stock- brokers may be required to collect information on a sam- ple of trades (value of trade and fee charged), and this information used to derive a current period fee schedule.

In the case of sales of shares, the underlying transaction represents the exchange of one asset for another (shares for cash).

Quantities underlying sales can be viewed similarly to share purchases (i.e.
some current period basket of consumption goods and services).

In reality, households review their investment strategies regularly in order to ‘‘store’’ their deferred consumption in whatever asset class they believe oﬀers the greatest security or prospect for growth.

A symmetrical treat- ment of the purchase and sale of shares is particularly appealing.

Unless diﬀerent fees or taxes apply to sales, there is no need to distinguish between the two in con- structing the index.
Deposit and loan facilities 10.137 Accounting for the costs of services provided by ﬁnancial intermediaries represents a signiﬁcant step up in complexity.

Even where a prior decision has been made to include such facilities within the scope of the CPI, the service being provided is diﬃcult to visualize comprehensively, and the prices comprise signiﬁcant elements that are not directly observable.
10.138 SNA 1993 recommends (6.125 and Annex III) that the value of ﬁnancial intermediation services output produced by an enterprise should be valued as the following sum: for ﬁnancial assets involved in ﬁnancial intermedia- tion, such as loans, the value of services provided by the enterprise to the borrower per monetary unit on account is the margin between the rate payable by the borrower and a reference rate; plus for ﬁnancial liabilities involved in ﬁnancial inter- mediation, such as deposits, the value of services provided by the enterprise to the lender or depositor per monetary unit on account is the margin between the reference rate and the rate payable by the enter- prise to the lender; plus the value of actual or explicit ﬁnancial intermediation service charges levied.
10.139 For a summary of the developments in national accounts treatment in this area, and a discussion of the notion of a reference rate, see OECD (1998).

In concept, SNA 1993 describes the reference rate as the risk-free or pure interest rate.

The value of the service provided to a borrower is the diﬀerence between the actual amount of interest paid by the borrower and the lower amount that would have been paid had the ref- erence rate applied.

The converse applies for depositors.


In practice, it is very diﬃcult to identify the reference rate, and in particular to avoid either volatility in or even negative measures of the value of such services (as would occur if the reference rate lay above the lending rate or below the deposit rate).
As a matter of practical expediency, an average of borrowing and lending rates may be used (with the mid-point being favoured).1 Given the complexities involved, expenditures on ﬁnancial intermediation required for index weighting purposes cannot be collected from households in expenditure surveys and so must be estimated by collecting data from ﬁnancial institutions.
10.140 In thinking about the construction of the index number, it is useful to start by considering the case of a traditional bank providing a single loan product and a single deposit product; the example will then be extended to a typical bank.

In some countries, the tra- ditional bank does not charge direct fees, but all income is derived through an interest margin on lending rates over deposit rates.
10.141 The base period weighting value of the ﬁnan- cial service (and so household consumption of such ser- vices) therefore is estimated by applying a margin (the absolute diﬀerence between the reference rate and the rate of interest charged to borrowers or paid to deposi- tors) to an aggregate balance (loan or deposit).
In line with the suggested treatment of other ﬁnancial transac- tions, the construction of accompanying price measures should allow for the indexation forward of base period balances, applying comparison period margins to cal- culate a money value.

The price index is then calculated as the ratio of comparison period and base period money values.

10.142 Again, the issue of an appropriate escalator needs to be addressed.
While the base period ﬂows of deposits and withdrawals can readily be conceptualized as forgone consumption at base period prices, how should the balances (stocks) reﬂecting an accumulation of ﬂows over a number of years be viewed?

If an age proﬁle for balances were available, accumulated con- sumption forgone could be computed as a moving average of the CPI.
The more practical alternative is to view base period balances as representing some quan- tum of consumption goods and services at base period prices, in which case the 12-month moving average CPI can be used.

This is consistent with the idea that house- holds investment decisions (and so accumulated ﬁnancial balances) on a regular basis, in this case annually.
review temporal consumption or 10.143 The traditional bank has all but disappeared in some countries and most ﬁnancial institutions now derive income from a combination of indirect fees (margins) and direct fees and charges, with the trend being for a move from margins towards direct fees.

In 1OECD (1998) expresses some concerns about the use of a mid-point reference rate as a measure of the risk-free rate of interest.
There are, however, some doubts about whether the conceptual ideal is for some ‘‘risk-free’’ interest rate, or whether a more appropriate concept might be the interest rate that would have been struck in the absence of ﬁnancial intermediaries (i.e.

the rate that would have been struck by depositors dealing directly with borrowers).

Such a rate would have incorporated the lenders’ knowledge of risk.
Taking the mid-point of the borrowing and lending rates would appear to be a good means of estimating this market-clearing rate.

SOME SPECIAL CASES 10.144 Given the clear scope for ﬁnancial this case, the challenge is to construct measures of price change that reﬂect the total price of the service and therefore capture any oﬀsets between margins and direct fees.
As with stockbroking services, there may also be taxes levied on ﬁnancial transactions or balances and these should also be included in the ‘‘price’’.

Frost (2001), for example, provides a description of the more practical aspects of constructing price indices for deposit and loan facilities based on recent Australian experience.
inter- mediaries to shift charges between the direct (fee) and indirect (margin) elements, there are clear dangers in constructing broad measures of margins – known by national accountants as ﬁnancial intermediation services indirectly measured (FISIM) – independent of direct fees and taxes.
Rather, the approach should be to construct price measures for speciﬁc (relatively homogeneous) products that can then be weighted together to provide a measure for deposit and loan facilities in aggregate, and taking account of both the direct and indirect elements in total price.

This represents a similar strategy to that adopted throughout the CPI.
For example, the index for motor vehicles is constructed by pricing a sample of individual vehicles and weighting these price measures to derive an aggregate, instead of, for example, attempting to directly construct an index for the supplier or pro- ducer of a range of vehicles.
10.145 The basic process is: ﬁrst, to select a sample of representative products from each sampled institution; second, to select a sample of customers for each product, and third, to estimate the total base period value of the service associated with each product by element (margin, direct fees and taxes).

These value aggregates can be viewed as being equivalent to prices for some quantum.
Comparison period prices are derived by moving for- ward the base period value aggregates as follows: Margin – index forward the base period balance and apply the comparison period margin (the diﬀerence between the comparison period reference rate and the product yield).

In practice, the ‘‘price’’ movement is given as the product of the indexation factor and the ratio of margins.
Fees – index forward the transaction values for each sampled account (or proﬁle) and apply the compar- ison period fee structure.

The ratio of new aggregate fees to base fees is used to move the fee value aggre- gate.

The aggregate fees in the base and comparison periods can be constructed as either arithmetic or geometric averages of the fees calculated for the individual customers.

 Taxes – as for fees, but use tax schedules instead of fee schedules.
10.146 Appendix 10.1 contains a worked example of the calculation of a price index for a single deposit product.

10.147 Since step function pricing and taxing sche- dules (for example, fees that are only payable after some number of transactions or if balances fall below some level) are prevalent in ﬁnancial services, samples of de- tailed customer accounts with all the necessary charging variables identiﬁed will be required.
These samples 197 CONSUMER PRICE INDEX MANUAL: THEORY AND PRACTICE should cover a full year’s activity.

If it is not possible to sample actual accounts, customer proﬁles may be developed as a fallback option.

10.148 To minimize problems associated with non- response and changing industry structures, a separate reference rate should be constructed for each sampled service provider.

The reference rate should be calculated in respect of all loans and deposits (including those to businesses).
Further, to avoid problems that may arise in the timing of accounting entries (e.g.

revisions, or interest income on credit cards), monthly yields, refer- ence rates and margins should be constructed by refer- ence to three-month moving averages of the reported underlying balances and interest ﬂows.

Real estate agency services 10.149 The services provided by real estate agencies in the acquisition and disposal of properties can be treated in a number of ways.
If the CPI is constructed as an economic cost of use index, these services are out of scope as they form part of the input costs of the notional landlords (SNA 1993 also assigns all transfer costs on dwellings to gross ﬁxed capital formation).

The transfer costs associated with the acquisition of a dwelling (legal fees, real estate agency fees and taxes) can be included in both a payments and an acquisitions CPI.
They can be classiﬁed as either a cost of home ownership or as a distinctly separate ﬁnancial service.

Although all transfer costs should be included in such measures, the discussion below focuses on real estate agents’ fees for simplicity.

Price measures for the other elements are calculated using similar procedures.

In all cases, the general ap- proach is to estimate the current cost of the various services relative to, and as they would apply to, some ﬁxed basket of activity in the base period.
Consistent with some of the areas already discussed, this involves indexing forward the base period expenditures on which the fees are charged (to preserve the underlying quan- tum) via some appropriate price index, and then esti- mating the fees payable in the comparison period.

10.150 Real estate agents typically quote their fees as some percentage of the price received for the dwelling.
In common with other items where charges are determined as a margin, this needs to be converted to a domestic currency price.

If the percentage margin is known, the agents’ price for any given transaction (sale/purchase of a dwelling for a known price) can be computed by mul- tiplying the value of the dwelling by the percentage margin, and the index can be constructed on the basis of estimates of both components.
10.151 The methodology chosen for estimating the percentage margin will depend upon an assessment of the variation in margins across and within individual agencies.

In the most straightforward case, ﬁrms may operate with a single percentage margin applicable to all transactions regardless of value.

In other words, at any point in time the percentage margins charged may vary by agency, but not by value of transaction within agency.
In this case, what is required is an estimate, in each 198 comparison period, of the average percentage margin charged by agencies.

This can be achieved by collecting the percentage margins, exclusive of any taxes levied on agents’ fees such as value added tax (VAT) or goods and services tax (GST), from a sample of agencies and deriving an average.

10.152 Percentage margins charged by individual agencies sometimes vary with transaction price (typi- cally declining with increasing prices of dwellings).
Where tariﬀs do vary within agencies, a more sophisti- cated estimation procedure may be required.

Using data from a sample of transactions from a sample of agents, the relationship between the value of transaction and the percentage margin can be derived through econometric analysis.

Empirical analysis will be required to deter- mine the precise functional form for this relationship.
For example, in the Australian case research has shown that ordinary least squares regression can be used to estimate this relationship and that the following func- tional form is adequate: R=a+b1ð1=pÞ+b2ð1=pÞ2 where: R=the commission rate, p=the house price, a=a constant, and b1 and b2 are parameters to be estimated.
10.153 Estimation of the current period value of transactions to which the percentage margin applies depends on whether real estate agency fees are classiﬁed as a cost of housing or as a separate ﬁnancial service.

If the former, the value of the current period transaction, relative to the value of the base period transaction, would reﬂect changes in house prices.
If the latter, where the purchase of a dwelling is regarded as forgone consumption, the current period value would reﬂect changes in the CPI itself.

10.154 If a single percentage margin is assumed to operate, then only a single current period transaction is required, i.e.

an estimate of the average value of base period transactions at comparison period prices.
For example, if real estate agency fees are classiﬁed as a housing cost, then the base period price is calculated by applying the average base period percentage margin to the average house price in the base period, with any VAT or GST then added.

The comparison period price is calculated by indexing forward the average base per- iod house price, applying the average comparison period percentage margin and adding GST or VAT.
10.155 If a single percentage margin is not assumed to operate, then a sample of representative base period transactions is required.

The monetary value of the margin on each representative transaction is then cal- culated from published tariﬀs or from an estimated functional relationship, such as that described above.

Comparison period prices are likewise estimated by ﬁrst indexing forward each of the base period representative transactions and then applying the same model.
Note that, in this case, there is no need to exclude any GST or VAT from the initial margins data.

Property insurance services 10.156 The construction of reliable price indices for insurance can be diﬃcult to achieve in practice.

This section is restricted to a discussion of property insur- ance, as this type of insurance can be assumed to operate in similar ways across countries.
It nevertheless provides only an illustration of the issues that index compilers face, with each sector raising speciﬁc conceptual and measurement diﬃculties.

For example, in the case of life insurance, insurance policies are often bundled with a long-term investment service yielding a ﬁnancial payout when insured persons survive the policy term.

Separa- tion of the service charges relating to the insurance and investment elements within a single premium poses sig- niﬁcant problems for index compilers.
10.157 For the purposes of the discussion below, property insurance is deﬁned to include: – dwelling insurance; – household contents insurance; – motor vehicle insurance.

10.158 The common feature of these policies is that for a fee (premium), households receive ﬁnancial com- pensation if a nominated event results in the loss of, or damage to, designated property.

The alternative to pur- chasing insurance is for the household to self-insure.
For households as a group, the service received is represented by the elimination of the risk of a ﬁnancial loss.

The appropriate treatment of property insurance in the CPI depends on whether the CPI is constructed using the acquisitions, use or payments approach.

Payments 10.159 Under the payments approach, each of the above policy types is in scope.
In thinking about how this property insurance should be included in the CPI, it is necessary to consider both the gross premiums payable and the claims receivable by households.

The deﬁnitions of gross premiums payable and claims receivable are straightforward.

It is possible, however, to treat claims receivable in a number of ways, which will have an impact on either the weight assigned to insurance or the weight assigned to the items insured.
Spending on insurance can be weighted on either a gross basis (i.e.

valued using gross premiums payable) or on a net basis (i.e.

valued using gross premiums payable less claims receivable).

Likewise, items which are insured against loss may also be weighted gross or net (in the latter case, excluding purchases explicitly ﬁnanced by insurance claims receivable).
Taken together, this suggests three plausible alternative treatments: – gross premiums, net expenditures; – net premiums, gross expenditures; – gross premiums, gross expenditures.

10.160 Gross premiums, net expenditures.

It may be argued that calculating expenditures net of purchases ﬁnanced by insurance claims avoids double counting of that portion of gross premiums which funds the claims.

There are some problems with this approach.


First, it is necessary to assume that all proceeds from insurance claims are used to purchase replacement items or to repair damaged items.

In some cases, claims receivable may be to compensate for damage or destruction to the SOME SPECIAL CASES property of agents beyond the scope of the index (e.g.

businesses, government or even other households where the CPI reference group covers only some subset of households).
Households may also choose to use the proceeds for entirely diﬀerent purposes.

Thus the esti- mation of the net expenditure weights is likely to involve some arbitrary choices.

More generally, because money is fungible, attempts to restrict coverage only to those expenditures made from selected sources of funds are questionable.

Finally, the potential distortion of weights for these items may reduce the usefulness of sub-indices for other purposes.

10.161 Net premiums, gross expenditures.


Within a payments index, the ‘‘net premiums, gross expenditures’’ approach is based on the view that claims receivable should be regarded as negative expenditure on insurance.

This may be seen as an attempt to avoid the double counting of expenditures on items ﬁnanced by claims receivable and already included in gross expenditures on other items elsewhere in the index.
The net premiums approach is much less problematic than the net expen- ditures approach (as at least the impact is restricted to the weights for insurance).

It may, however, be argued that the net premiums approach is inconsistent with approaches adopted for other items in a payments index, in particular mortgage interest and consumer credit charges, where weights are based on gross payments.
Any allowance for interest receipts would be likely to yield negative weights since households are generally net savers overall.

10.162 The fact that the net premiums approach eﬀectively measures the value of the insurance service as required for indices constructed according to both the acquisitions and use approaches is incidental.

The task here is to determine the appropriate treatment for a payments-based index.

10.163 Gross premiums, gross expenditures.


The ‘‘gross premiums, gross expenditures’’ approach is based on the view that the claims receivable by households simply represent one of the sources of funds from which expenditures are made.

This is the most appealing ap- proach for a payments index, as it recognizes the fungible nature of money and provides a consistent means of identifying both the item coverage of the index and the relative weights by reference only to the actual outlays of households.
Use 10.164 Under the use approach, dwelling insurance is out of scope as an input cost of the notional landlord.

The weights should relate to the value of the insurance service consumed by households.

This is deﬁned as being equal to: gross insurance premiums payable by house- holds, plus premium supplements, less provisions for claims, less changes in actuarial reserves.

10.165 It is not possible to estimate the nominal value of the net insurance service from household expen- diture surveys alone.
For weighting purposes, the most appealing approach is to obtain data from a sample of insurance providers, permitting estimation of the ratio of net insurance services to gross premiums, and to apply 199 CONSUMER PRICE INDEX MANUAL: THEORY AND PRACTICE this ratio to the estimated value of gross premiums obtained from household expenditure surveys.

However, it has not been possible to devise a corresponding price measure that is conceptually sound.
For this reason, those countries that have adopted the net concept for weighting purposes are using movements in gross insurance premiums as a proxy price measure.

Acquisitions 10.166 Under the acquisitions approach, all three items are in scope.
Because the objective is to measure price inﬂation for the household sector, the expenditures required for weighting purposes should reﬂect the insur- ance companies’ contribution to the inﬂationary process, which equates to the value of the insurance service as per the use approach.
Pricing gross insurance premiums 10.167 The gross insurance premium payable by households in any one period is determined by the con- ditions of the policy, the administration costs and proﬁt objectives of the insurance provider, the risk of a claim being made and any relevant taxes.

For any single policy, the principal quality-determining characteristics (gen- erally speciﬁed in the conditions of the policy) can be summarized as being: – the type of property being covered (dwellings, motor vehicles, etc.
); – the type of cover provided (physical damage, liability, etc.

); – the nature of the compensation (replacement cost, current market value, etc.

); – any limits on the amount claimable; – the location of the property; – amount of any excess payable by the insured; – risks (or events) covered.

10.168 While it is clear that pricing to constant quality requires these conditions to be held ﬁxed, there is also a question about whether the risk of a claim being made should be held constant.
In other words, if the incidence of, say, vehicle theft increases, should this be regarded as a quality improvement or simply a price change?

If, on the one hand, it is argued that as the consumers’ decision to insure is based on their assess- ment of the likelihood of suﬀering a loss compared to the premium charged, the risk factors should be held constant.

On the other hand, it may be argued that, once insured, the consumer simply expects to be compensated for any loss.
From the perspective of the consumer, any increase in risk simply represents an increase in the insurer’s cost base (which may or may not be passed on to the consumer by way of a price change).

Obtaining data of suﬃcient reliability to make quality adjustments in response to changes in risk is problematic, so in practice most indices reﬂect changes in risk as a price change.
10.169 In pricing insurance policies, the approach should be to select a sample of policies representative of 200 those policies held in the base period and to reprice these in subsequent periods.

Taking dwelling insurance as an example, base period insurance policies would be taken out to insure dwellings of various values and types (e.g.

timber or brick) in diﬀerent locations.


The price samples should therefore consist of speciﬁcations that aim to cover, in aggregate, as many combinations of these variables as is reasonable.

While the conditions of the policy, the dwelling type and location should be held constant over time, the value of the dwelling should be updated each period to reﬂect changes in house prices (i.e.

the underlying real quantity needs to be preserved).
It is important to note that, as the premiums will be related in some way to the value of the insured property, the price index for insurance can change without there being any change in premium schedules.

10.170 Every eﬀort should be made to identify any changes in the conditions applying to selected policies in order to facilitate appropriate quality adjustments.
Examples would include cessation of coverage for spe- ciﬁc conditions and changing the excess (or deductible) paid by the consumer when a claim is made.

Estimates of the value of such changes may be based on the insurance company’s own assessments of their likely impact on the value of total claims payable.
If it is assumed that the change in the aggregate value of claims can be equated to the change in service to the consumer (compared to the service that would have been provided prior to policy renewal), then an appropriate adjustment can be made to the premium to provide a (quality- adjusted) movement in price.

For example, consider the case where the excess on a policy is doubled and advice from the company is that this will result in a 3 per cent drop in the aggregate value of claims payable.
This could be considered as equivalent to a 3 per cent increase in price.

Using gross premiums as a proxy for the net insurance service 10.171 The net insurance service charge captures the administration costs and proﬁts of the insurance provi- der along with any taxes.

The problem is that taxes on insurance are normally levied on the gross premiums.
Therefore, if the gross insurance premiums are subject to a high rate of tax, then the taxes will account for an even higher proportion of the net insurance service charge.

Simply using the gross insurance premium inclusive of taxes as the price measure understates the real eﬀect of any increase in the tax rates.

This is best illustrated by way of an example.

10.172 For the sake of simplicity, assume that there are no premium supplements and no actuarial reserves.
Then the insurance service charge is given by gross premiums less provisions for claims.

Suppose the only change between two periods is a change in the tax rate – from 5 per cent of gross premiums to 20 per cent.

Then the values in Table 10.10 are likely to be observed.

Under this scenario it is clear that the insurance service charge has increased from $45 to $60 (an increase of 33.3 per cent), yet gross premiums have only increased by 14.3 per cent.
Table 10.10 Illustration of the impact of taxes on measures of insurance services ($) Period Premiums before tax 1 2 100 100 Tax 5 20 Gross premiums Claims Insurance service 105 120 60 60 45 60 10.173 Given that changes in the tax rates on gross insurance premiums are often subject to signiﬁcant vari- SOME SPECIAL CASES ation, this is a non-trivial problem.
A practical solution is to decompose insurance service into two components – insurance service before tax (or net of tax) and tax on insurance services.

The price measure for the ﬁrst is constructed by reference to movements in gross pre- miums net of tax, and the price measure for the second is given by changes in taxes on gross premiums.

Further research is required to develop a workable methodology for directly measuring changes in prices of insurance services before tax.
201 CONSUMER PRICE INDEX MANUAL: THEORY AND PRACTICE Appendix 10.1 Calculation of a price index for a deposit product (a) Base period sample account.

Only a single month’s data is used in this example.

In practice, many accounts would be sampled with each account containing data for a full year.
Taxes Date 2 Jan 12 Jan 13 Jan 13 Jan 14 Jan 14 Jan 16 Jan 16 Jan 16 Jan 19 Jan 19 Jan 19 Jan 19 Jan 20 Jan 21 Jan 21 Jan 22 Jan 22 Jan 22 Jan 23 Jan 23 Jan 27 Jan 27 Jan 27 Jan 27 Jan 28 Jan 28 Jan 29 Jan 30 Jan 30 Jan 30 Jan Total taxes Debit (D) or Credit (C) Transaction Transaction value ($) Tax ($) Balance ($) Over the counter withdrawal Deposit EFTPOS1 transaction Over the counter withdrawal Own ATM2 cash Own ATM cash Own ATM cash Over the counter withdrawal Cheque Own ATM cash Own ATM cash Deposit Cheque EFTPOS transaction Other ATM cash Cheque Cheque Cheque Cheque Over the counter withdrawal Cheque Own ATM cash Cheque Cheque EFTPOS transaction Over the counter withdrawal Cheque Cheque Over the counter withdrawal Cheque Cheque 1EFTPOS (Electronic Funds Transfer Point Of Sale).
2ATM (Automatic Teller Machine).

Fees Activity Over the counter withdrawal EFTPOS transaction Own ATM cash Own ATM cash Cheque Deposit Total fees Total no.

No.

charged Amount($) 6 3 6 1 13 2 2 0 0 1 3 2 6.00 0.00 0.00 1.20 3.00 0.00 10.20 Fees and taxes are calculated using data in tables (b) and (c), respectively.

Source: Woolford (2001) 202 (b) Fee schedule.

This is a summary of the information typically available from ﬁnancial institutions.
For each period, the table includes the number of free transactions and the per transaction charge for additional transactions.

A zero number free indicates that no transactions are free and a zero charge indicates that all transactions are free.

SOME SPECIAL CASES Description Over the counter withdrawal EFTPOS transaction Own ATM cash Other ATM cash Cheque Deposit Source: Woolford (2001).

Base period No.

free 4 10 10 0 10 0 Charge ($) 3.00 0.50 0.50 1.20 1.00 0.00 Current period No.
free 4 9 9 0 9 0 Charge ($) 3.00 0.50 0.50 1.20 1.00 0.00 (c) Tax schedule.

This is a table of tax rates of the type that used to be employed in Australia.

The debits tax is levied on all debit transactions to eligible accounts, with the amount charged being set for ranges of transaction values (i.e.

using a step function).

Financial institutions duty is levied on all deposits, the amount being determined as a percentage of the value of the deposit.

Max.
Max.

1 100 500 5 000 10 000 Bank accounts debit tax Transaction value ($) Min.

0 1 100 500 5 000 10 000+ Financial institutions duty (%) Base period 0.06 Source: Woolford (2001).

Tax ($) Base period 0.00 0.30 0.70 1.50 3.00 4.00 Current period 0.00 0.30 0.70 1.50 3.00 4.00 Current period 0.06 (d) Interest data.

The table presents, in summary form, the balances and annualized interest ﬂows derived by taking moving averages of data reported by ﬁnancial institutions.
Interest rates and margins are calculated from the balances and ﬂows.
Base period Current period Balance ($ million) Interest ($ million) Interest rate (%) Margin (%) Balance ($ million) Interest ($ million) Interest rate (%) Margin (%) Deposit products Personal accounts Current accounts Other accounts Business accounts Total deposit accounts Loan products Personal accounts Business accounts Total loan accounts Reference rate Source: Woolford (2001).
203 CONSUMER PRICE INDEX MANUAL: THEORY AND PRACTICE (e) CPI data.

The table presents data required to derive the indexation factor.

This example follows the Australian practice of a quarterly CPI.

If a monthly CPI is produced, 12-term moving averages would be required.

All groups 4-term moving average Indexation factor (movement) Source: Woolford (2001).

t5 117.5 t4 121.2 t3 123.4 t2 127.6 122.4 t1 129.1 125.3 1.0237 (f) Projected current period sample account.
The opening balance and transaction values are derived by applying the indexation factor to the base period amounts.

The tax payable is determined by reference to the data in table (c).

Fees payable are determined by reference to the data in table (b).
Debit (D) or Credit (C) Transaction Transaction value ($) Tax ($) Balance ($) Over the counter withdrawal Deposit EFTPOS transaction Over the counter withdrawal Own ATM cash Own ATM cash Own ATM cash Over the counter withdrawal Cheque Own ATM cash Own ATM cash Deposit Cheque EFTPOS transaction Other ATM cash Cheque Cheque Cheque Cheque Over the counter withdrawal Cheque Own ATM cash Cheque Cheque EFTPOS transaction Over the counter withdrawal Cheque Cheque Over the counter withdrawal Cheque Cheque Total No.
No.

charged Amount ($) 6 3 6 1 13 2 2 0 0 1 4 2 6.00 0.00 0.00 1.20 4.00 0.00 11.20 Taxes Date 2 Jan 12 Jan 13 Jan 13 Jan 14 Jan 14 Jan 16 Jan 16 Jan 16 Jan 19 Jan 19 Jan 19 Jan 19 Jan 20 Jan 21 Jan 21 Jan 22 Jan 22 Jan 22 Jan 23 Jan 23 Jan 27 Jan 27 Jan 27 Jan 27 Jan 28 Jan 28 Jan 29 Jan 30 Jan 30 Jan 30 Jan Total taxes Fees Activity Over the counter withdrawal EFTPOS transaction Own ATM cash Own ATM cash Cheque Deposit Total fees Source: Woolford (2001).
204 SOME SPECIAL CASES (g) Indices for current accounts.

This table brings the results together.

The current period value aggregates are derived as follows.

For margins – the base period aggregate is multiplied by the product of the indexation factor (e) and the ratio of the current and base period margins for current accounts (d).

For fees – the base period aggregate is multiplied by the ratio of total fees payable on the sample account in the current period (f ) and the base period (a).
For taxes – the same procedure is followed as for fees.

Component Margins Fees Taxes Total Source: Woolford (2001).

Base period Value aggregate ($) 28 344 11 904 14 739 54 987 Index 100.0 100.0 100.0 100.0 Current period Value aggregate ($) 27 913 13 071 14 818 55 803 Index 98.5 109.8 100.5 101.5 205 ERRORS AND BIAS Introduction 11.1 This chapter discusses the general types of potential error to which all price indices are subject.
The literature on consumer price indices (CPIs) discusses these errors from two perspectives, and this chapter presents the two perspectives in turn.

First, the chapter describes the sources of sampling and non-sampling error that arise in estimating a population CPI from a chapter sample of observed prices.

Second, reviews the arguments made in numerous recent studies that attribute bias to CPIs as a result of insuﬃciently accurate treatment of quality change, consumer sub- stitution and other factors.
It should be emphasized that many of the underlying issues discussed here are dealt with in much greater detail elsewhere in the manual.

the Types of error 11.2 One of the main objectives of a sample survey is to compute estimates of population characteristics.

Such estimates will never be exactly equal to the popu- lation characteristics.

There will always be some error.

Table 11.1 gives a taxonomy of the diﬀerent types of error.
See also Balk and Kersten (1986) and Dale´ n (1995) for overviews of the various sources of stochastic and non-stochastic errors experienced in calculating a CPI.

Two broad categories can be distinguished: sampling errors and non-sampling errors.

Sampling error 11.3 Sampling errors are due to the fact that an estimated CPI is based on samples and not on a complete enumeration of the populations involved.

Sampling errors vanish if observations cover the complete popula- tion.
As mentioned in previous chapters, statistical oﬃ- ces usually adopt a ﬁxed weight price index as the object Table 11.1.

A taxonomy of errors in a consumer price index Total error: Sampling error Selection error Estimation error Non-sampling error Observation error Overcoverage Response error Processing error Non-observation error Undercoverage Non-response 11 of estimation.

A ﬁxed weight index can be seen as a weighted average of partial indices of commodity groups, with weights being expenditure shares.
The estimation procedures that most statistical oﬃces apply to a CPI involve diﬀerent kinds of samples.

The most important kinds are: for each commodity group, a sample of commodities to calculate the partial price index of the commodity group; for each commodity, a sample of outlets to calculate the elementary price index of the commodity from individual price observations; a sample of households needed for the estimation of the average expenditure shares of the commodity groups.
(Some countries use data from national accounts instead of a household expenditure survey to obtain the expenditure shares.)

11.4 The sampling error can be split into a selection error and an estimation error.

A selection error occurs when the actual selection probabilities deviate from the selection probabilities as speciﬁed in the sample design.

The estimation error denotes the eﬀect caused by using a sample based on a random selection procedure.
Every new selection of a sample will result in diﬀerent elements, and thus in a possibly diﬀerent value of the estimator.

Non-sampling error 11.5 Non-sampling errors may occur even when the whole population is observed.

They can be subdivided into observation errors and non-observation errors.

Observation errors are the errors made during the process of obtaining and recording the basic observations or responses.
11.6 Overcoverage means that some elements are included in the survey which do not belong to the target population.

For outlets, statistical oﬃces usually have inadequate sampling frames.

In some countries, for instance, a business register is used as the sampling frame for outlets.

In such a register, outlets are classiﬁed accord- ing to major activity.

The register thus usually exhibits extensive overcoverage, because it contains numerous outlets which are out of scope from the CPI perspective (e.g.
ﬁrms that sell to businesses rather than to house- holds).

In addition, there is usually no detailed infor- mation on all the commodities sold by an outlet, so it is possible that a sampled outlet may turn out not to sell a particular commodity at all.
11.7 Response errors in a household expenditure survey or price survey occur when the respondent does not understand the question, or does not want to give the 207 CONSUMER PRICE INDEX MANUAL: THEORY AND PRACTICE right answer, or when the interviewer or price collector makes an error in recording the answer.

In household expenditure surveys, for example, households appear to systematically underreport expenditures on commodity groups such as tobacco and alcoholic beverages.
In most countries, the main price collection method is by persons who regularly visit outlets.

They may return with prices of unwanted commodities.

11.8 The price data are processed in diﬀerent stages, such as coding, entry, transfer and editing (control and correction).

At each step mistakes, so-called processing errors, may occur.

For example, at the outlets the price collectors write down the prices on paper forms.
After the collectors have returned home, a computer is used as the input and transmission medium for the price information.

It is clear that this way of processing prices is susceptible to errors.

11.9 Non-observation errors are made when the intended measurements cannot be carried out.

Under- coverage occurs when elements in the target population do not appear in the sampling frame.
The sampling frame of outlets can have undercoverage, which means that some outlets where relevant commodities are purchased cannot be contacted.

Some statistical oﬃces appear to exclude mail order ﬁrms and non-food market stalls from their outlet sampling frame.

11.10 Another non-observation error is non-response.

Non-response errors may arise from the failure to obtain the required information in a timely manner from all the units selected in the sample.
A distinction can be drawn between total and partial (or item) non-response.

Total non-response occurs when selected outlets cannot be contacted or refuse to participate in the price survey.

Another instance of total non-response occurs when mail questionnaires and collection forms are returned by the respondent and the price collector, respectively, after the deadline for processing has passed.
Mail questionnaires and collection forms that are only partially ﬁlled in are examples of partial non-response.

If the price changes of the non-responding outlets diﬀer from those of the responding outlets, the results of the price survey will be biased.

11.11 Total and partial non-response may also be encountered in a household expenditure survey.

Total non-response occurs when households drawn in the sam- ple refuse to cooperate.
Partial non-response occurs, for instance, when certain households refuse to give infor- mation about their expenditure on certain commodity groups.

Measuring error and bias Estimation of variance 11.12 The variance estimator depends on both the chosen estimator of a CPI and the sampling design.

Boon (1998) gives an overview of the sampling methods that are applied in the compilation of CPIs by various Euro- pean statistical institutes.
It appeared that only four of them use some sort of probability techniques for outlet selection, and only one uses probability sampling for item selection.

In the absence of probability techniques, 208 so-called judgemental and cut-oﬀ selection methods are applied.

11.13 In view of the complexity of the (partially connected) sample designs in compiling a CPI, an inte- grated approach to variance estimation appears to be problematic.
That is, it appears to be diﬃcult to present a single formula for measuring the variance of a CPI, which captures all sources of sampling error.

It is, how- ever, feasible to develop partial (or conditional) mea- sures, in which only the eﬀect of a single source of variability is quantiﬁed.
For instance, Balk and Kersten (1986) calculated the variance of a CPI resulting from the sampling variability of the household expenditure survey, conditional on the assumption that the partial price indices are known with certainty.

Ideally, all the conditional sampling errors should be put together in a unifying framework in order to assess the relative importance of the various sources of error.
Under rather restrictive assumptions, Balk (1989a) derived an inte- grated framework for the overall sampling error of a CPI.

11.14 There are various procedures for trying to estimate the sampling variance of a CPI.

Design-based is, variances of Horvitz– variance estimators (that Thompson estimators) can be used, in combination with Taylor linearization procedures, for sampling errors arising from a probability sampling design.
For instance, assuming a cross-classiﬁed sampling design, in which samples of commodities and outlets are drawn inde- pendently from a two-dimensional population, with probabilities proportional to size (PPS) in both dimen- sions, a design-based variance formula can be derived.

In this way Dale´ n and Ohlsson (1995) found that the sampling error for a 12-month change of the all- commodity Swedish CPI was of the order of 0.1–0.2 per cent.
11.15 The main problem with non-probability sampling is that there is no theoretically acceptable way of knowing whether the dispersion in the sample data accurately reﬂects the dispersion in the population.

It is then necessary to fall back on approximation techniques for variance estimation.

One such technique is quasi-randomization (see Sa¨ rndal, Swensson and Wretman (1992, p. 574)), in which assumptions are made about the probabilities of sampling commodi- ties and outlets.
The problem with this method is that it is diﬃcult to ﬁnd a probability model that ade- quately approximates the method actually used for outlet and item selection.

Another possibility is to use a replication method, such as the method of random groups, balanced half-samples, jackknife, or bootstrap.

This is a completely non-parametric class of methods to estimate sampling distributions and standard errors.

Each replication method works by drawing a large number of sub-samples from the given sample.
From interest can be each sub-sample the parameter of estimated.

Under rather weak conditions, it can be shown that the resulting esti- mates approximates the sampling distribution of the original estimator.

For more details on the replication methods see Sa¨ rndal, Swensson and Wretman (1992, pp.

418–445).

the distribution of Qualitative descriptions of non-sampling errors 11.16 It is still more diﬃcult to obtain quantitative measures of the non-sampling errors.
Thus the use of qualitative indications is the only possibility.

For instance, the coverage of the sampling frames as a proxy of the target populations can be addressed (including gaps, duplications and deﬁnitional problems).

The per- centage of the target outlet samples from which re- sponses or usable price data were obtained (i.e.

the response rates) can be provided.


Any known diﬀerence in the prices of responding outlets and non-respond- ing outlets can be described, as can an indication of the method of imputation or estimation used to com- pensate for non-response.

Several categories of non- sampling errors provide the bulk of the bias issues discussed below.

Procedures to minimize errors 11.17 The estimation error can be controlled by means of the sampling design.
For example, by increas- ing the sample size, or by taking selection probabilities proportional to some well-chosen auxiliary variable, the error in the estimated CPI can be reduced.

The choice of an adequate sampling design for the CPI is an extremely complex matter.

The target population is the set of all goods and services that are acquired, used or paid for by households from outlets in a particular time period.
A proper probability sampling procedure selects a sample by a random mechanism in which each good or service in the population has a known probability of selection.

In combination with a Horvitz–Thompson estimator, such a probability sampling design will produce an index that is (approximately) unbiased and precise.
11.18 The following three probability sampling designs are used extensively in survey practice: simple random (SI) sampling, probability proportional to size (PPS) sampling, and stratiﬁed sampling with SI or PPS sampling per stratum.

The advantage of SI sampling is its simplicity; it gives each population element the same probability of being included in the sample.

PPS sam- pling has the advantage that the more important ele- ments have a larger chance of being sampled than the less important ones.
For instance, at Statistics Sweden the outlets are selected with probabilities proportional to some proxy for size, namely their number of employees.

Unequal probability designs can lead to a substantial variance reduction in comparison with equal probability designs.

In stratiﬁed sampling, the population is divided into non-overlapping sub-populations called strata.
For instance, at the United Kingdom Oﬃce for National Statistics the population of outlets is split by outlet type (multiple, independent or specialist) to form diﬀerent strata.

In each stratum a sample is selected according to a certain design.

One of the reasons why stratiﬁed sampling is so popular is that most of the potential gain in precision of PPS sampling can be captured through stratiﬁed selection with SI sampling within well- constructed strata.
Stratiﬁed sampling is in several aspects simpler than PPS sampling.

ERRORS AND BIAS 11.19 Because appropriate sampling frames are samples are frequently obtained by non- lacking, probability methods.

Judgemental (or expert choice) sampling is one form of non-random selection.

In this case an expert selects certain ‘‘typical’’ elements where data are to be collected.

With skill on the part of the expert a fairly good sample might result, but there is no way to be sure.
A more sophisticated non-probability method is quota sampling.

In quota sampling the pop- ulation is ﬁrstly divided into certain strata.

For each stratum, the number (quota) of elements to be included in the sample is ﬁxed.

Next the interviewer in the ﬁeld simply ﬁlls the quotas, which means in the case of out- let sampling that the selection of the outlets is ulti- mately based on the judgement of the price collectors.
Another non-probability method is cut-oﬀ sampling, which means that a part of the target population is delib- erately excluded from the sample selection process.

In particular, this procedure is used when the distribution of the value of some auxiliary variable is highly skewed.

For instance, a large part of the population may consist of small outlets whose contribution to total sales is modest.

A decision may then be taken to exclude from the sam- pling frame the outlets with the lowest sales.
Because the selection is non-random, non-probability methods usually lead to more or less biased estimates.

Empirical results of research undertaken by Statistics Netherlands nevertheless show that non-probability selection meth- ods do not necessarily perform worse, in terms of the mean square error, than probability sampling techniques (De Haan, Opperdoes and Schut, 1997).
11.20 Provided that the sampling design is given, the sampling variance of an estimated (all-commodities) CPI can in general be lowered by: – enlarging the samples of households, commodities and outlets; – the application of suitable stratiﬁcations to the var- ious populations (e.g.

grouping commodities with respect to similarity of price changes).
11.21 It is important to allocate optimally the avail- able resources both between and within the diﬀerent CPI samples, since badly allocated samples may lead to unnecessarily high sampling errors.

The Swedish variance estimation results, presented in Dale´ n and Ohlsson (1995), show that the error resulting from commodity sampling is relatively high compared with the error resulting from outlet sampling.
In this case, it is worth- while increasing the sample size of commodities and reducing the sample size of outlets.

11.22 A systematic analysis of sampling errors oﬀers possibilities for improving or reducing cost.

The pro- blem of optimum sample allocation is usually for- mulated as the determination of the sizes of the samples of commodities and outlets, and their distribution over the strata that minimizes the sampling error of an all- commodities CPI, subject to the available budget.
11.23 As already mentioned, a business register is usually not an adequate sampling frame for outlets, because it provides extensive overcoverage.

It is recom- mended to set up an appropriate sampling frame by enumeration of the main outlets within each sampled 209 CONSUMER PRICE INDEX MANUAL: THEORY AND PRACTICE municipality.

Such enumeration yields a list of all outlets in a municipality together with the commodity groups that belong to their assortments.
A less expensive way to organize an outlet sampling frame is to ask the price collectors – who may be assumed to know the local situation well – to make a list of outlets where purchases are made by households.

11.24 The populations of commodities (and vari- eties) and outlets are continually changing through time.

The composition of most commodity groups is not constant over time, because commodities disappear from the market and new ones appear.
The passage of time also plays a disturbing role with respect to the outlet pop- ulation: outlets close, temporarily or permanently; new outlets emerge; the importance of some outlets dimin- ishes or increases.

The samples of commodities (and varieties) and outlets should be reviewed and updated periodically to maintain their representativity with re- spect to the current buying habits of the households.
11.25 Response errors caused by the underreporting of certain categories of household expenditure can be adjusted by using producer-based estimates from the national accounts (see Linder (1996) for an example).

Measurement errors by price collectors can be reduced by providing them with hand-held computers for data entry.

In this way the validation of observed prices can be executed at the point of price collection (i.e.
in the out- let), by means of an automatic comparison of the cur- rently observed price quote with the previously observed one (by setting a limit on the percentage price change) and with the price quotes obtained from other outlets (by setting suitable upper and lower limits).

Details are provided by Haworth, Fenwick and Beaven (1997).

11.26 It is useful to appoint data collection super- visors to conduct quality assurance checks on the data collectors.
It is also a good idea to organize regularly meetings where price collectors and statisticians from the head oﬃce can share their experiences.

In this way, the statisticians will keep in touch with the conditions in the ﬁeld, and may take the opportunity to provide more information about frequently made price collection errors and new representative goods.

11.27 It is important to check the collected price data for processing errors and, where possible, to correct these errors.
This activity is called data editing.

When editing is carried out on individual observations, it is called micro- editing.

When the resources to spend on data editing must be minimized, while at the same time maintaining a high level of data quality, selective editing and macro- editing are possibilities.

Selective editing is a form of traditional micro-editing, in which the number of edits is kept to a minimum.

Only those edits which have an impact on the survey results are carried out.
Macro- editing oﬀers a top-down approach.

The edits are carried out on aggregated data (for instance, the price index numbers of a commodity group) instead of individual records (for example, price observations).

Micro-editing of individual records is then carried out only if macro- edits raise suspicion.

In particular, attention should be paid to outliers among the observations.

11.28 Non-response usually introduces selection bias.
There are three methods for the treatment of missing 210 price observations.

First, the corresponding price can be excluded from the data set of previous prices, so that the set of previous prices is ‘‘matched’’ with the set of current prices.

Second, this matching can be achieved by using an imputed (or artiﬁcial) price for the missing one.
The imputed price can be calculated by either carrying for- ward the previous price observation or by extrapolating the previous price observation using the change of other price observations for the same commodity.

Third, there is the possibility to reweight the sample.

The objective of reweighting is to inﬂate the weight given to the prices of the responding outlets.

This compensates for those prices that are lost by non-response.
11.29 In a household expenditure survey, missing data are usually imputed with the help of information on the same household from a previous observation period or other households from the same observation period.

To reduce bias in the average expenditure pattern arising from selective non-response, a household expenditure survey sample of households is generally post-stratiﬁed by a number of household characteristics, such as income, composition and size.
Types of bias 11.30 This section reviews several categories of error, either in pricing or in index construction, that potentially can lead to bias in the overall CPI.

The emphasis here is on the categorization of errors, along with some con- sideration of their likely size, rather than on methods to reduce or eliminate the errors.
The question might arise of why such a discussion is necessary, since such issues as quality change, and the appropriate methods for hand- ling them in the CPI, are dealt with at both a conceptual and operational level in other chapters.

11.31 The reason this chapter addresses the topic of CPI bias per se is the great surge in interest in price measurement problems during the mid-1990s.
Especially in the United States, the view became widespread that the CPI was subject to systematic upward biases because of the failure to deal adequately with consumer sub- stitution, product quality improvements, and the intro- duction of new items and services.
Moreover, it was recognized, ﬁrst, that the existence of such upward bias would have fundamental implications for the measure- ment of recent trends in output and productivity, and second, that the elimination of upward bias could sub- stantially improve the government budget situation through reduced government expenditures and increased tax revenues (see, for example, Eldridge (1999) and Duggan and Gillingham (1999)).
These discoveries led to a series of papers and reports on CPI measurement problems, often accompanied by point estimates of aggregate bias.
11.32 Prominent examples of these quantitative stu- dies of bias are those by the Advisory Commission to Study the CPI (United States Senate, 1996), Con- gressional Budget Oﬃce (1994), Crawford (1998), Cunningham (1996), Dale´ n (1999a), Diewert (1996c), Lebow, Roberts and Stockton (1994), Lebow and Rudd (2003), Shapiro and Wilcox (1997b), Shiratsuka (1999), White (1999), and Wynne and Sigalla (1994).

Responses and estimates by statistical agencies include those pro- vided by Abraham et al.
(1998), US Bureau of Labor Statistics (1998), Ducharme (1997), Edwards (1997), Fenwick (1997), Lequiller (1997), Moulton (1996b), and Moulton and Moses (1997).

Among the many other discussions of the CPI bias issue are those reported by Baker (1998), Boskin et al.

(1998), Deaton (1998), Die- wert (1998a), Krueger and Siskind (1998), Nordhaus (1998), Obst (2000), OECD (1997), Pollak (1998), Pop- kin (1997), and Triplett (1997).
11.33 Two points are worth making at the outset with respect to measuring bias in CPIs.

First, the issue has usually been addressed in the context of the cost of living index (COLI).

That is, the CPI bias has been deﬁned as the diﬀerence between the rate of increase in the CPI and the rate of increase in a true COLI.

Many authors on bias have taken as given that the COLI should be the CPI’s measurement objective.
Somewhat diﬀerent conclusions might be reached if the index objective were taken to be a pure price index.

Notably, the gains in consumer welfare from a widening array of new goods, or the ability of consumers to substitute away from items with increasing relative prices, might be deemed irrelevant and an index that ignored those fac- tors might not be judged biased on that account.
11.34 The second point is that CPI bias is not amenable to estimation with the same level of rigour as that used in CPI variance estimation.

Since the COLI or other ideal target index is unobserved, analysts have been forced to rely in part on conjectures and on general- izations from fragmentary empirical evidence in order to quantify the extent of bias.
The notable exceptions are with respect to substitution bias, when traditional Laspeyres indices and indices using superlative formulae can be computed using the same underlying price and expenditure data, and the diﬀerences construed as a measure of the upward bias from use of the Laspeyres formula.

11.35 Several diﬀerent taxonomies of bias have appeared in the literature mentioned above.
It is suﬃ- cient, however, to employ four categories roughly cor- responding to those set forth in the best-known study, namely the Final report of the Advisory Commission to Study the CPI (the Boskin Commission), established by the United States Senate Finance Committee in 1995.

These categories are: upper-level substitution bias; ele- mentary aggregate bias; quality change and new goods bias; and new outlet bias.
11.36 These categories can be further broken down into two subgroups according to whether they refer to errors in individual price measurements or errors in com- puting index series.

Quality change bias and new goods bias arise because of failures to measure adequately the value to consumers of individual goods and services that appear in (or disappear from) the marketplace.

It should be recognized that discussions of ‘‘new goods’’ problems apply equally to all products, whether goods or services.
At a conceptual level, it can be diﬃcult to distinguish these two biases from each other.

Operationally, how- ever, quality change bias pertains to the procedures for comparing new products or models with the older pro- ERRORS AND BIAS ducts they replace in the CPI samples.

In general, new goods bias can be thought of as applying to wholly new types of products, or products that would not enter samples routinely through forced replacement.
New outlet bias, sometimes referred to as outlet substitution bias, is similar to new goods bias but is focused on the appearance of new types of stores or marketing methods that oﬀer goods at lower prices or higher quality.

11.37 The other categories of bias refer to the pro- cedures for constructing index values from component series.

As noted throughout this manual, CPI construc- tion can be thought of as taking place in two steps, or at two levels.
At the lower level, individual price quotations are combined; at the upper level, these basic indices are aggregated together.

Corresponding to these two levels are two forms of potential bias.

Elementary aggregate bias involves the averaging formulae used to combine price quotations into basic indices.

Upper-level substi- tution bias applies to the formulae used to combine those elementary aggregates into higher-level indices.
These components of potential bias, and the means used to measure them, are discussed in more detail below.

Components of bias Upper-level substitution bias 11.38 Upper-level substitution bias is perhaps the most widely accepted source of CPI bias, and the kind with which economists are most familiar from textbook expositions of price index theory and practice.
Simply stated, it arises when CPIs employ the Laspeyres for- mula (see Chapter 17), which is well known to provide an upper bound on a cost of living index under certain assumptions about consumer behaviour.

As noted in paragraph 11.34 above, quantitative measures of upper- level substitution bias can be generated by comparing Laspeyres price indices to Fisher ideal, To¨ rnqvist or other superlative indices.
Under certain assumptions about, for example, constant preferences, these will stand as relatively precise bias estimates.

11.39 Genereux (1983) and Aizcorbe and Jackman (1993) provide such index comparisons and estimates of upper-level substitution bias using actual CPI index series for Canada and the United States, respectively.

Other early studies by Braithwait (1980) and Manser and McDonald (1988) estimate the substitution bias in Uni- ted States national account indices.
In lieu of superlative indices, the Braithwait study uses estimated exact cost of living indices based on demand system estimation.

A similar estimate for the Netherlands is provided by Balk (1990).

In these studies, the existence of an upward bias from the Laspeyres formula is demonstrated con- sistently.
The biases in the annual index changes in indi- vidual years are relatively small, averaging 0.1 to 0.3 percentage points, and depend empirically on such fac- tors as the distance from the Laspeyres base period, the level of index detail at which the alternative formulae are applied, and whether the superlative index is of the ﬁxed base or chained variety.
11.40 The major diﬀerences between Laspeyres and superlative indices derive from the variation in relative 211 CONSUMER PRICE INDEX MANUAL: THEORY AND PRACTICE prices over the period being compared, and from the shift in quantities consumed towards those index categories that have fallen in relative price.
This leads to several conclusions: If index movements are characterized by continuing, uniform drift in relative prices over time, with accom- panying drifts in consumption, the size of the annual Laspeyres bias will tend to increase with the distance from the base period.

(Greenlees (1997) notes, how- ever, that there is little evidence for this phenomenon in the United States; see also Szulc (1983).)
Under the same circumstances, reducing the expendi- ture weight chaining interval will work to reduce the upper-level substitution bias in the Laspeyres CPI.

The more frequent chaining will increase the weight given to indices that are falling in relative price, thereby reducing the rate of CPI growth.

Conversely, if there is ‘‘bouncing’’ in relative index movements, frequent chaining can lead to an upward ‘‘chain drift’’ in a Laspeyres index.
Upper-level substitution bias will tend to be larger during periods of higher inﬂation, if these periods also have greater relative price variation.

Little empirical evidence exists on this point, however.

11.41 The concept of upper-level substitution bias has been derived and discussed in the context of cost of living index theory, but an equivalent bias may be deﬁned from the perspective of the pure price index.
If the Fisher ideal or other superlative index is judged preferable on the basis of its symmetric treatment of base period and current period expenditure patterns, then the diﬀerence between that index and a Laspeyres could be interpreted as a measure of representativity bias.

A similar argument could be applied with respect to lower-level substitution bias within elementary index cells.
11.42 Recently, Lebow and Rudd (2003) have deﬁned and estimated another category of bias related to upper- level aggregation.

They concluded that the consumer expenditure survey weights used in the United States CPI were subject to error because of, for example, under- reporting of alcohol and tobacco expenditures.

This will lead to a weighting bias if the errors in relative weight are correlated with component index changes.
(Sources for, and problems in, expenditure weight estimation are dis- cussed in detail in Chapter 4.)

Elementary aggregate bias 11.43 Elementary aggregate bias can be divided into two components: formula bias and lower-level sub- stitution bias.

An elementary index in the CPI is biased if its expectation diﬀers from its measurement objective.
The term formula bias (or functional form bias) is used here to denote a situation in which the elementary index formula has an upward bias relative to the pure price index.

When the measurement objective is a cost of living index, the elementary index formula suﬀers from lower-level substitution bias (or within-stratum sub- stitution bias) if it does not reﬂect consumer substitution among the items contained in that index cell.
Thus, given any elementary index formula, the two forms of 212 bias can be distinguished according to the objective of the elementary index.

11.44 Chapters 9 and 20 of this manual discuss the characteristics of alternative elementary index formulae.

A key result is that the Carli formula for the arithmetic average of ratios has an upward bias relative to the trend in average item prices.
Consequently, Eurostat has prohibited use of this formula in computations for the Harmonized Indices of Consumer Prices (HICPs).

The weighted formulae used in basic indices of the United States CPI had some characteristics of the Carli formula prior to procedural and computational changes made in 1995 and 1996.

The problems and the methods chosen to address them are discussed, for example, by Reins- dorf and Moulton (1997) and Moulton (1996b).
11.45 The ratio of arithmetic averages (Dutot) and geometric mean (Jevons) formulae eliminate formula bias as deﬁned here, and both are permitted by Eurostat.

Their expectations diﬀer, however, when item prices do not change at a uniform rate.

The diﬀerences provide one way of evaluating the potential importance of lower-level substitution bias.
The geometric mean formula is exact for a cost of living index if consumers follow the Cobb– Douglas behavioural model, whereas the formula based on the ratio of arithmetic averages corresponds to zero- substitution behaviour.

Thus, if the goal is to approx- imate a cost of living index, the geometric mean formula is likely to be judged preferable.
11.46 In the future, scanner data may make it pos- sible to record item-level consumption data at a daily, weekly or monthly frequency and to use those data in superlative index calculations.

Currently, however, it is impossible to employ superlative formulae to compute elementary CPI indices.

Some assumption, such as the Cobb–Douglas, must be made in order to approximate a cost of living index.
Note that the substitution that the index ideally should reﬂect involves consumer choice among all the items in the cell: diﬀerent products, prod- ucts in diﬀerent outlets, diﬀerent package sizes of the same product, or the same product oﬀered for sale at diﬀerent times of the period to which the index applies (see Dalton, Greenlees and Stewart (1998)).

Thus, the appropriate degree of assumed substitution behaviour should depend, in principle, on the dimensions of variety within the item category.
11.47 The method used by the statistical agency for sampling items within a category will determine the eﬀec- tiveness of formula choice in dealing with lower-level substitution bias.

For example, if only a single repre- sentative item is chosen to represent the category, the index formula will fail to reﬂect the consumer response to any relative price change in the universe of items.
More generally, the geometric mean formula index suﬀers from an upward bias in small samples, so lower-level sub- stitution bias may be underestimated in empirical com- parisons of the geometric mean to other index formulae.

White (1999) discusses the relationship between sampling error and bias estimates.

See also McClelland and Reinsdorf (1999) on the small sample bias in the geo- metric mean.
11.48 The impact of formula choice can be estimated with some degree of precision over a given historical period.

Any corresponding bias, however, can be esti- mated only by assuming that the geometric mean or other functional form successfully approximates the index’s measurement objective.
11.49 As implied by the above discussion, the impor- tance of elementary aggregate bias will vary by country, depending on the particular index formulae used, the degree of heterogeneity within index strata, and the sampling methods employed.

Also, as with upper-level substitution bias, elementary aggregate bias will vary level of inﬂation in the economy if with the overall absolute and relative price changes are correlated.
11.50 The performance of any formula for elemen- tary aggregate calculation will also be aﬀected by the methods used by the statistical agency to handle special situations, such as seasonal goods and other products that are temporarily unavailable.

Armknecht and Maitland-Smith (1999) discuss how the failure to impute missing prices can lead to bias in the modiﬁed Laspeyres and other index formulae.
Quality change and new products bias 11.51 Discussion of potential CPI biases arising from inadequate quality adjustment has a long history.
For example, the Stigler Committee report on United States price statistics (Price Statistics Review Committee, 1961) indicated that ‘‘if a poll were taken of professional economists and statisticians, in all probability they would designate (and by a wide majority) the failure of the price indices to take full account of quality changes as the most important defect of these indices’’.
In most studies of bias, unmeasured or mismeasured quality change is also the largest contributor to the total estimated bias.

Just as quality adjustment is widely recognized as an extremely diﬃcult process, however, it is correspondingly diﬃcult to measure any quality change bias.

11.52 Unlike substitution bias, which can be esti- mated by comparison of alternative formulae, quality change bias must be analysed on a product-by-product basis.
Products and their associated index components will experience widely varying rates of quality change over time.

Moreover, the methods used for quality adjustment will also vary.

Whereas the linking method may dominate in terms of frequency of use, important index components may employ production cost, hedo- nic adjustment, or the other methods described in Chapters 7 and 21.
11.53 A crucial point to recognize is that the direc- tion of overall quality change does not imply the direc- tion of any quality change bias.

Non-experts sometimes assume that the CPI does little or no quality adjustment, and that it therefore must overestimate price change in view of the many demonstrable improvements over time in the quality of goods and services.
Rather, for any component index, the issue is whether the direct or indirect method chosen for quality adjustment over- estimates or underestimates the relative quality of replacement items in the CPI sample.

The resulting bias can be either positive or negative.

11.54 Empirical evidence on quality change bias has been based largely on extrapolation from individual stu- ERRORS AND BIAS dies of particular products.
These individual studies may involve, for example, comparisons of hedonic regression indices to the corresponding CPI series or estimates of the value of some product improvement that is ignored in CPI calculations.

Although the majority of such studies have suggested upward rather than downward bias, the reliance on fragmentary evidence has led to criticism by observers who point to evidence of quality declines that have not been subjected to systematic analysis.
11.55 Especially for services, overall quality trends can also be a matter of subjective valuation.

New tech- nology has led to unambiguous improvements in the quality of many consumer durables and other goods.

By contrast, in service sectors such as mail delivery, public transport and medical care, it can be diﬃcult to evaluate changes in quality.
Airline travel, for example, has be- come safer and faster but perhaps less comfortable and reliable in recent decades, and the lack of cross-sectional variation in these characteristics makes the use of hedonic quality adjustment problematic.

11.56 New product bias, like elementary aggregate bias, can be divided conceptually into two components.

The ﬁrst concerns the failure to bring new products into the CPI sample with suﬃcient speed.
This can lead to upward bias if those new products later experience large price reductions that are not reﬂected in the index.

The second component is the welfare gain that con- sumers experience when a new product appears.

This may not be viewed as a bias, however, when the cost of living index is not accepted as the CPI’s measurement objective.
11.57 As discussed in Chapter 8, ‘‘new goods’’ can be: products that replace predecessor items, for example CDs replacing vinyl records and tapes; product varieties that widen the range of consumer choice, such as imported beers and ethnic restaurants; or products that represent wholly new categories of consumption, such as microwave ovens or mobile telephones.

11.58 Like quality change bias, new product bias has sometimes been estimated primarily by generalization from individual product evidence.
A frequent approach has been to measure the price change for a product or category during a period prior to its entry into the CPI sample.

Studies by Hausman (1997, 1999) of breakfast cereals and cellular telephones provided quantitative measures of the consumer surplus gain from the new products, but this complex econometric approach has not been applied widely.

Some of the Boskin Commis- sion’s estimates of new product bias, notably those for food, were necessarily based on conjecture.
11.59 Also, like quality change bias, new product bias could be negative if the range of products decreases, if valuable consumer goods disappear from the market, or if the index fails to capture phases of rapid price increase for items.

Most observers, however, seem to agree on the direction of bias as upward, and that the uncertainty concerns the magnitude.

New outlet bias 11.60 Conceptually, new outlet bias is identical to new product bias.
It arises because of the failure to reﬂect 213 CONSUMER PRICE INDEX MANUAL: THEORY AND PRACTICE either price changes in new outlets not yet sampled, or the welfare gain to consumers when the new outlets appear.

The explanation for its existence as a separate bias category is twofold.

The ﬁrst reason is historical: new outlet bias was identiﬁed by Reinsdorf (1993) as a potentially major explanation for anomalous movements in the United States CPI.
Second, the methods used to sample and compare outlets diﬀer from those used with products, and the problems in controlling new outlet bias are somewhat diﬀerent.

11.61 A failure to maintain a current outlet sample can introduce bias because the new outlets are distinctive in their pricing or service policy.

Reinsdorf (1993), for example, focused on the growth of discount stores.
It should be noted, however, that the problem could also be geographical in nature; it is important to employ outlet sampling frames that reﬂect new as well as traditional shopping locations.

11.62 One way that new products enter the CPI sample is through forced replacement, when exiting or less successful products disappear from shelves.

Outlet disappearance is less frequent, and agency procedures may not provide for automatic replacement.
Moreover, when a new outlet enters the sample there are no stan- dard procedures for comparing data at the new and old outlets.

Thus, the index will not incorporate any eﬀects of, for example, lower price or inferior service quality at the new outlet.

11.63 Reinsdorf (1993) estimated the degree of new outlet bias by comparing average prices at outlets entering and disappearing from United States CPI samples.
There has been little or no empirical work, however, on the measurement or consumer valuation of outlet quality.

As a consequence, there is little evidence on which to evaluate the accuracy of new outlet bias estimates.

Summary of bias estimates 11.64 The 1996 Boskin Commission report gave a range of estimates for the total upward United States CPI bias of 0.8 to 1.6 percentage points, with the point estimate being 1.1 percentage point.
This total reﬂects the straightforward summation of the component bias estimates.

As reported by the United States in United States General Accounting Oﬃce (2000), however, changes in CPI methods subsequent to 1996 led the Boskin Commission members to reduce their estimates of total bias.

Lacking evidence to the contrary, additivity of biases has been assumed in most such studies.
Shapiro and Wilcox (1997b) provide probability distributions and correlations of their component bias estimates, yielding an overall conﬁdence interval for the total bias.

Most detailed studies of bias also conclude that the CPI bias is in an upward direction, although there have been numerous criticisms of that conclusion.

11.65 It is apparent that statistical agencies cannot compute or publish CPI bias estimates on a regular basis.
Many of the same obstacles that prevent the elimination of bias also stand in the way of estimating bias.

These include the lack of complete data on product-level con- sumer preferences and spending behaviour, and the inability to observe and value all diﬀerences in quality among items in the marketplace.

Without such infor- mation it is impossible to calculate a true cost of living index, and similarly impossible to measure the diver- gence between its rate of growth and the growth rate of the CPI.
11.66 Statistical agencies have been reluctant to provide their own estimates of CPI bias.

In some cases, they have accepted the existence of substitution bias, recognizing that the use of a Laspeyres formula implies that the CPI usually will overstate price change relative to a cost of living index.

Statistical agencies have, how- ever, been reluctant to draw even qualitative conclusions from the fragmentary and speculative evidence on quality change, new products and new outlet bias.
Conclusion 11.67 In order to ensure public conﬁdence in a CPI, a detailed and up-to-date description of the methods and data sources should be published.

The document should include, among other things, the objectives and scope of the index, details of the weights, and last but not least, a discussion of the accuracy of the index.

A description of the sources and magnitude of the sampling and non- sampling errors (coverage, non-response rates, etc.)
in a CPI provides users with valuable information on the limitations that might apply to their uses of the index.

One example of a handbook of CPI methods is that published by the United States Bureau of Labor Statis- tics (1997), which devotes a section to the varieties and sources of possible error in the index.

214 ORGANIZATION AND MANAGEMENT 12 Introduction 12.1 Consumer price indices (CPI) are one of the most important and widely used of macroeconomic indicators.
As well as informing economic policy, they are used for indexation of welfare beneﬁts, pensions, gilts and sec- urities, and also for escalation clauses in private con- tracts.

Accuracy and reliability are paramount for a statistic as important as a CPI.

12.2 The process of producing a consumer price index needs to be carefully planned.

Individual circum- stances vary to such an extent that this manual cannot be too prescriptive about timetables or critical path analysis of all the steps involved.
Figure 12.1 nevertheless pro- vides an outline of the kind of schedule of activities that should result from a detailed examination of the logistics of the whole periodic operation of data collection and the computation of the index.

12.3 The guidance given in this chapter, which is based on the experiences of a number of national sta- tistical institutes, presents a range of organizational options.
As individual circumstances can vary, the exam- ples given of good practice may be for some oﬃces aspirational.

12.4 In reviewing these options, this chapter covers the relationships between the ﬁeld and central oﬃce (which kind of work is carried out in central oﬃce, the ﬂow of information between each part of the organiz- ation, etc.).
The size, frequency, cost or complexity of the collection of prices as the basis of the index may mean that in some countries not all these operations and relationships will be appropriate.

The use of both a central and a local collection, or outsourcing of certain elements of the collection, may not always be eﬀective.

If the index is compiled infrequently, from a relatively small number of outlets, or concentrates on only speciﬁc location types, diﬀerent circumstances will demand dif- ferent solutions.
Local collection 12.5 A local price collection involves collectors visiting individual outlets to collect prices for a variety of goods and services.

This is the predominant method of price collection in most countries.

The range and number of outlets visited and the types of goods and services priced will vary between countries.
12.6 Although the precise method of local price collection will vary, each price collector will usually be responsible for collection from a certain location or from certain types of outlet.

Collectors will visit the same outlets in each collection period to attempt to price the same items.

Through this type of arrangement, price collectors are able to build up eﬀective relationships with retailers and specialist knowledge.
12.7 There are a number of important criteria relating to the conduct of the collection, whether the national statistical institute uses its own staﬀ or contracts out the collection (as discussed below).

These criteria include: Collectors should always be smartly dressed and polite – whoever employs them, they are representing the national statistical institute.

 They should carry identiﬁcation to conﬁrm their role and status.
They should make themselves known to the retailer or store manager when they arrive, and before they begin collecting prices.

 They should comply with any request from the shop- keeper whenever possible, for instance, if the store is very busy and the shopkeeper asks the collector to return later in the day.

 The collection should be carried out as quickly as possible, causing minimal disruption to store business.
12.8 Collectors should also follow rules of common sense in preparing for a collection.

These may include making sure that they have: spare pens; the appropriate forms; a clipboard; a local map; spare batteries (if the collection is computerized); money for shopping centre car-parks; and wet weather clothing, if appropriate.

In some circumstances a mobile telephone will also be useful.
Contracting out 12.9 One of the decisions facing any statistical agency carrying out a price collection is whether to use in-house staﬀ or to tender the collection to an external organization, such as a private market research com- pany, another part of the agency, or another govern- mental department that specializes in surveys.

12.10 The nature of the price collection and the distribution and proﬁle of statistical staﬀ may help to determine whether the collection is suitable for con- tracting out.
Where price collection is continuous, or involves complicated decision-making (such as quality adjustment), or where prices are collected from a small number of locations, it may be advantageous to keep the collection in-house.
However, if the collection takes place over a small number of days per month, from a large number of locations, is relatively straightforward and involves routine or simple decision-making (perhaps selecting from a prespeciﬁed list of codes), then con- tracting out may be considered if there are enough 215 CONSUMER PRICE INDEX MANUAL: THEORY AND PRACTICE Figure 12.1 Price collection procedures Is the outlet open?

No Is it permanently shut?

Yes No Yes Yes Is the outlet willing to cooperate?


Is refusal because authorized person is not available?

No Try again later if the instructions say so, otherwise report the facts.

Yes Is the product type to be priced currently available to purchase?

No Report the non-availability and the reason for it.

Yes Yes No If the variety is neither seasonal nor expected to be available to purchase again next month, then: Yes Is there a large difference from the price recorded last month?
Is this because offinal clearance for damaged or soiled items?

Disregard the price.

No No Report the facts to HQ Price next item.

Report the non- availability and the reason for it, providing the full description of the unavailable variety if central office does not already have it.

Select another outlet according to instructions.

Report it.

Yes Is the collector authorized to select a replacement outlet?

No State the reason, e.g.


sale, special offer, black market price, replacement outlet, replacement item.

Is the collector authorized to find a replacement?

No Is the replacement the same kind as its predecessor?

No Yes Yes Select another variety likely to remain available.

Record a description of it in sufficient detail both to cover quality differences and to enable exact identification.

Are both prices available for the same month?


No Estimate and report the amount of price difference reflecting the value of the quality differences revealed by the descriptions of the original variety and its replacement.

Yes Yes Is the collector authorized to make quality judgements?

No Report the facts to HQ.

Record the price and, if it is non-standard, also record the weight, size or quantity.

No Is the outlet likely to be permanently shut next month?
Yes Is the variety expected to become permanently unavailable next month?

No Price next item.

Yes 216 market research companies with suitable skills existing in the country.

12.11 Contracting out local collection can lead to reduced costs.

Where price collection is carried out electronically, the responsibility for purchase and main- tenance of data capture devices may be transferred to the contractor.
12.12 Contracting out may also allow statistical staﬀ to spend more time analysing data rather than collecting them.

By divorcing the role of data collector and data checker, statistical staﬀ may feel more comfortable questioning the validity of price data.

Accuracy of data collected can be directly linked to the performance of the contractor through performance measures which drive incentive payments (and penalties if targets are not achieved).
Central collection 12.13 Central shop-collected prices are prices obtained from the central oﬃces of major retailing chains with national pricing policies.

Branches of these chains may be excluded from the local collection if data can be collected more eﬀectively centrally.

Data suppliers may provide information on paper forms, or by entering price data on spreadsheets and forwarding them to the national statistical institute by email, CD-ROM or on ﬂoppy disks.
Mail order catalogues can also be treated as central shops: prices are recorded as and when the catalogues are issued.

These prices are then combined with those for the same items from the local collection.

12.14 Price data for services or fees may be collected centrally from organizations such as trade associations, national or local governmental departments and so on.
Whenever possible, these prices are obtained from one central source, although there will have to be contact with regional or competing companies if there are local variations.

Data may be requested in writing or by tele- phone, or may come automatically because the national statistical institute is on a provider’s mailing list.
Provi- ders may send either a full price list or tariﬀ sheet, from which the relevant prices will be extracted by the CPI staﬀ, or just the prices of those items speciﬁed in the data request.

All price quotations should be conﬁrmed by some form of written documentation.

Frequency of enquiry varies across the range of items and depends on when prices are known or expected to change.
The most common frequencies are monthly or quarterly, but there are also instances of collecting as and when necessary, although in these cases checks must be in place to ensure that all price data are reported.

For instance, this may be the case where tariﬀs for gas, electricity and water change once a year on a predetermined date.

Quality in the ﬁeld 12.15 Quality is an important aspect of price collec- tion.
A high-quality price collection enables a statistical agency to have conﬁdence in the index it produces, and ensure that observed price changes are genuine and not the result of collector error.

It is important that pro- cedures are developed to ensure that the standard of ORGANIZATION AND MANAGEMENT collection is maintained at a high level for every collec- tion period.
These procedures will form the basis of collector training and should be included in any training material developed for price collectors.

Guidance to collectors should cover price index principles, organiz- ational issues and validation procedures.

Descriptions 12.16 Accurate item descriptions are vitally impor- tant in ensuring item continuity.

Collectors’ descriptions should be comprehensive enough to ensure that collec- tors are able to price the same item in each collection period.
It is therefore important that contributors record the attributes which uniquely deﬁne the item they are pricing.

For example, for clothes it will be important that colour, size and fabric composition are speciﬁed to ensure that the same item is priced each month.

For fresh fruit and vegetables, useful attributes to record may be country of origin, class and variety.
12.17 Accurate item descriptions will assist the price collector and head oﬃce in choosing a replacement for an item that has been withdrawn and will also help to identify changes in quality.

Head oﬃce staﬀ should be encouraged to spend some time, each collection period, going through collectors’ descriptions to ensure that the correct items are being priced.

Collectors should also be encouraged to review their descriptions to ensure that they contain all of the relevant information.
It may be useful to ask collectors occasionally to switch collections with another collector so that they understand the importance of good descriptions.

Continuity 12.18 Continuity is one of the most important prin- ciples of price collection.

Because a price index measures price changes, it is vital that the same item is priced every month in order to establish a true picture of price changes.
So if, for example, a jar of a supermarket’s own brand of strawberry jam has been selected, that partic- ular brand and ﬂavour should continue to be collected.

If it is out of stock in the collection period, another brand and ﬂavour should not be used.

If, however, in sub- sequent collection periods, the selected jam continues to be out of stock, but another ﬂavour of the same brand and price is available, then this item should be chosen as a comparable item and the item description suitably amended.
If no comparable item exists, then a new item must be chosen, and the description amended.

Thus a new price chain will begin.

It is not possible to be pre- scriptive because the concept of equivalence will vary between diﬀerent countries; but for practical purposes it is important that a detailed description of the items being priced is kept.
12.19 As continuity is so important in the compila- tion of an accurate price index, collectors should be encouraged to check with the retailer that an item is out of stock before replacing it.

Some guidelines may be drawn up by the head oﬃce of the national statistical institute to cover diﬀerent items.
Food items, for exam- ple, will usually come back into stock in the follow- ing collection period, and so should not be replaced 217 CONSUMER PRICE INDEX MANUAL: THEORY AND PRACTICE immediately, whereas fashion clothing will rarely come back into stock once a ‘‘season’’ has ended or the stock has been exhausted, and so should be replaced imme- diately in the collection.
12.20 Collectors should also be encouraged to plan their route for price collection to take account of outlet opening and closing times and any special retailer requests.

Collectors may ﬁnd it useful to compile a route map, listing the order in which outlets should be visited.

This is particularly useful when the collection in a loca- tion has to be undertaken by a diﬀerent collector, for example to cover for sick leave.
Collectors should be encouraged to try to collect prices at similar times within each collection period.

This is particularly important when pricing volatile items, such as petrol and oil, where there can be sharp ﬂuctuations.

Data entry queries 12.21 Once the price data are correct and complete, a series of validation checks may be run.

In deciding on what checks should be carried out, account should be taken of the validation checks carried out in the ﬁeld.
For example, the use of hand-held computers will increase the potential for validation at time of price collection and reduce the need for detailed scrutiny at head oﬃce.

In addition, it would clearly not be productive or cost- eﬀective to repeat tests already carried out.
12.22 The range of tests carried out may include: Price change: The price entered is compared with the price for the same product in the same shop in the previous month, and triggers a query where the price diﬀerence is outside preset percentage limits.

These limits vary, depending on the item or group of items, and may be determined by looking at historical evi- dence of price variation.
If there is no valid price for the previous month, for example because the item was out of stock, the check can be made against the price two months or three months ago.

 Maximum/minimum prices: A query is raised if the price entered exceeds a maximum or is below a minimum price for the item of which the particular product is representative.

The range may be derived from the validated maximum and minimum values observed for that item in the previous month expan- ded by a standard scaling factor.
This factor may vary between items, again based on previous experience.

12.23 If a hand-held computer is used, both of these tests can be implemented easily to take place at the time of collection, otherwise they will need to be conducted in the head oﬃce as soon as possible after collection and before prices are processed on the main system.
A failure in either test should not result in the collector being unable to price the item, but should prompt the collector to check and conﬁrm the entry, and prompt for an explanatory comment.

12.24 Queries raised may be either dealt with at head oﬃce or sent to the price collector for resolution.

For example, scrutiny of a form might show that a big price diﬀerence has arisen because the item priced was a new product replacing another that has been discontinued.
In this case there may be no need to raise a query with the 218 price collector unless there is evidence to suggest that to label the item ‘‘new product’’ is incorrect.

12.25 Where an error is discovered too late in the process to resolve, head oﬃce will need to reject it and exclude that item from that month’s index.

Care should be taken that the item is also excluded from the base month so that the basket is kept constant.
Feedback 12.26 Collectors should be encouraged to give feed- back to head oﬃce on their experiences of price collect- ing.

Collectors are a valuable source of information and often give good early feedback on changes in the market- place.

Collectors can often warn of size or product changes before the head oﬃce is able to derive this information from other sources such as trade magazines.

Collectors’ feedback can be used to support observed price movements and to provide supplementary brieﬁng material.
It can also form the basis of a newsletter for collectors.

Quality checks in local collection: The role of auditors 12.27 The whole periodic routine of collecting prices in the ﬁeld needs to be carefully planned and monitored, with arrangements in place to reﬂect local conditions.

Circumstances vary, so it is not appropriate to be too prescriptive.

It is, however, important to ensure that price collectors send in information when it is due.
If they do not do so, it is necessary to ﬁnd out the reason and to take appropriate action.

It is also important to check that the information sent in is accurate and complete.

12.28 One way of monitoring the work being carried out by price collectors is to employ auditors to occa- sionally accompany collectors during the ﬁeld collection, or to carry out a retrospective check on the data that have been collected.
Monitoring 12.29 If an auditor intends to accompany a price collector, he or she will need to inform the collector in advance in order to arrange meeting details.

In general, the auditor will not accompany the whole price collection but will spend a few hours observing the price collection in a speciﬁc location.
For example, it may be desirable to observe the collection of certain items or in particular outlets where collection might be problematical, and the price collector may need to rearrange his or her route accordingly.

12.30 Prior to monitoring, the auditor will need to carry out preparation work – a pre-monitoring check.

Such a check might involve looking at descriptions, prices, price history and indicator codes of the items collected in the chosen location.
This type of check will enable the auditor to have a good idea of the standard of collection prior to going into the ﬁeld and may suggest on which areas of the collection the auditor should concentrate his or her eﬀorts.

12.31 An auditor’s main duty is to ensure that the price collector is following the procedures and instruc- tions laid down for price collection and that the collec- tion is being performed competently.
While the auditor may not have the role of a trainer, the opportunity may be taken to give some coaching when errors are noted.

There should also be the opportunity for the collector to ask the auditor relevant questions during the monitoring exercise.

12.32 Auditors may undertake other duties at a loca- tion besides accompanying the collector.

For instance, they may enumerate outlets or carry out an item review.
Following a monitoring visit, the auditor should compile a report detailing the observations made while accom- panying the collector.

This report should include a summary of ﬁndings, a list of points for action, and a recommended course of action.

Auditors may advise that a collector receive extra training on certain aspects of the price collection; head oﬃce (or the contractor, if the collection has been outsourced) should act on this.
This report will then be used as a starting point on the audi- tor’s next visit.

In other instances, general problems may arise where solutions need to be disseminated to all price collectors, perhaps by issuing revised instructions or through a newsletter.

Backchecking 12.33 Another approach to monitoring the standard of price collection is to carry out a backcheck, a retro- spective check of a proportion of the prices recorded during the collection.
12.34 Backchecks can be used to: – assess the standard of competence of individual price collectors; – audit the overall standard of price collection; – identify general training needs or the speciﬁc needs of an individual; – highlight any key issues including, for example, pro- blems with documentation or instructions issued by head oﬃce; – identify areas where collection is problematical; for example all collectors may have problems in certain types of outlets, prompting the need for more detailed head oﬃce instructions.
12.35 Backchecking should be done by an expert independent of the process (preferably employed by the national statistical institute).

Backchecking is carried out by visiting the selected outlet and re-collecting the prices and other relevant information, such as attribute or description codes.

This activity should be carried out close to the original collection period to avoid problems of price changes occurring in the interim.
It is important that backcheckers seek permission from the shopkeeper beforehand and follow the general criteria of conduct for local collection, as described in paragraphs 12.5 to 12.12.

12.36 For a backcheck to be a useful exercise it is important that performance criteria are determined to which all backcheck results can be compared.

These criteria should set, for example, the acceptable number of price errors per number of items checked.
Well-deﬁned ORGANIZATION AND MANAGEMENT criteria will enable easy identiﬁcation of a poorly per- forming collector or location following a backcheck.
12.37 A backcheck may include a range of tests to identify the following: price diﬀerence – if the price is diﬀerent, the auditor should check with shopkeepers to see if there has been a price change since the original collection took place; insuﬃcient item description – each item should be uniquely deﬁned so that another collector can step in to cover the absence of the usual collector, for example in the event of illness; wrong item priced – such as incorrect size being chosen; items wrongly recorded as missing or temporarily out of stock.
12.38 A report should be sent to head oﬃce for scrutiny once the backcheck has been completed.

Head oﬃce will then need to take appropriate action, which may include, for example, retraining or sending out supplementary instructions.

Other auditor functions 12.39 The range of tasks that an auditor carries out will vary from one statistical agency to another.

Mon- itoring the standard of price collection will always be the main focus of the auditor.
There are a number of other areas, however, in which auditors can be called upon to contribute.

12.40 Auditors may be required to help with the sampling of locations and items.

Auditors can check that proposed collection locations contain an adequate range of shops.

They can also advise on economic conditions in these locations and on any dangerous areas.

Auditors can carry out commodity work.


For example, if a par- ticular item seems to be causing diﬃculty for price col- lectors, auditors can speak to collectors and retailers with a view to determining reasons for these diﬃculties.

Auditors can also advise on changes to basket compo- sition.

They can ensure that products suggested by head oﬃce are available across the country, and can suggest item descriptions and weight bands.
Furthermore, auditors can provide reports on collection in existing locations.

For example, head oﬃce may raise a query about a particular outlet in a particular location; audi- tors can visit this outlet to ﬁnd the answer to the question or to persuade a retailer to continue with the survey.

Quality checks in head ofﬁce 12.41 Four kinds of regular checking are necessary in head oﬃce: to ensure that the price collectors’ reports are sent in when they are due.
If this is not done, it is necessary to ﬁnd out the reason and to take appropriate action to obtain the reports; to conﬁrm that the reports contain what they are supposed to contain, i.e.

that ﬁelds which must be ﬁlled in have not been left blank, that numeric ﬁelds contain numbers and that non-numeric ﬁelds do not; 219 CONSUMER PRICE INDEX MANUAL: THEORY AND PRACTICE to review and edit each return.
Substitutions may have to be made centrally or those made by the collectors may have to be approved.

Unusual (or simply large) price changes may need to be queried.

Items priced in multiple units or varying weights may have to be converted to price per standard unit.
Missing prices must be dealt with according to standard rules relat- ing to the cause; errors introduced when keying the numbers into the computer or transcribing them onto worksheets must be found and corrected, and preferably avoided in the ﬁrst place by eliminating the need to transcribe.
12.42 It should be noted that the way the data are organized in worksheets or in the computer may diﬀer from the way they are organized on receipt, since they will arrive at the central oﬃce organized by collector, outlet and item.

Their origin should, however, be recorded so that reference to it can be made should processing disclose any problems with the data.
Fur- thermore, even if codes provided to the collectors to list items and to describe or qualify the prices are used unchanged in the processing, other codes may have to be used for information which comes in from the collectors in non-coded form.

12.43 How the checking is organized will vary from country to country.

In some cases, local or regional supervisors will do some of it; in other cases, it will be more appropriate for it all to be done centrally.
Some of these tasks can be done by computer, others manually.

Therefore, no general suggestion can be made about the sequence of the work or about its division into diﬀerent parts.

12.44 Procedures should be in place to check that all documents, messages or ﬁles are returned from the ﬁeld so that price collectors can be contacted about missing returns.

Initial checks should then be carried out to ensure that data are complete and correct.
For instance, checks should be run to ensure that unexpected duplicate prices (i.e.

for the same item, in the same shops, in the same location) are not taken on, and that the location, outlet and item identiﬁer codes which accompany each price exist and are valid.

If any prices fail these checks, a query should be raised with the price collector for clar- iﬁcation.
Since some of the checking may require refer- ence back to the price collectors (or to their supervisors or respondents when direct mail questionnaires are used), the timetable for producing the index must allow for this communication to take place.

12.45 Following the checks that the price data are correct and complete, a series of validation checks may be run.

In deciding on what checks should be done, account should be taken of the validation checks carried out in the ﬁeld.
The use of hand-held computers will increase the potential for validation at the time of price collection and reduce the need for detailed scrutiny at head oﬃce.

It would clearly not be productive or cost- eﬀective to repeat all the tests already carried out locally, except as a secondary audit or random check that those checks have been completed.

12.46 The range of checks that might be carried out is covered in paragraphs 12.21 to 12.25.
In addition, it is 220 possible for the head oﬃce to use the price data received that month to identify outliers.

Reports 12.47 Reports should routinely be generated for most representative items, to help the analyst pick out particular prices for which the level or change stands out as diﬀerent from that reported for similar varieties else- where, or simply where the change lies outside certain speciﬁed limits.
Thus, a computer printout can list all prices which either fall well outside the range of prices obtained last time for that representative item, or for which the percentage change from last time for the same item in the same outlet falls outside a speciﬁed range.

The limits used will vary from item to item and can be amended in the light of experience.
The analyst can then work through the printout, ﬁrst ascertaining whe- ther there has been a keying-in error, and then examin- ing whether any explanation furnished by the collector adequately explains the divergent price behaviour or whether a query should be sent back to the supervisor or collector.

The timetable should allow for this, and anomalous observations should be discarded where an acceptable explanation or correction cannot be obtained in time.
12.48 Other reports may be produced regularly on the basis of reports for several periods (e.g.

several months) to detect accumulated patterns, thus enabling broader problems to be detected.

For example: One collector’s reports might show many more ‘‘outlet closed’’ remarks than those of other collectors, per- haps indicating either a motivational or training need on the part of that collector, or a change in retail trade patterns in a particular area.
Variety substitution for a particular representative item might become more numerous than hitherto, suggesting a possible need for revision of the speciﬁ- cation or the choice of another representative item.

 Where tight speciﬁcations list a number of brands and models of which one is to be chosen, but a large number of prices are for items not speciﬁed in the original list, this will suggest that the speciﬁed brands and models are no longer appropriate and a review of the list is required.
The dispersion of price changes for a particular representative item might be much larger than it used to be, raising the question of whether it has been appropriately speciﬁed.

12.49 Routine computer-generated reports should enable those in charge of the index to detect the exis- tence of all such problems.

Two types of reports are particularly useful: index dispersion reports, and price quote reports.

12.50 Index dispersion report.


This is a list of items indicating the current index for each item, the number of valid quotes for each item, and the number of price relatives (the ratio of current price to previous valid price) in each of a series of pre-selected ranges (for example, less than 40, 40–49, .

.

.

190–199, greater than 199).

The index dispersion reports can be used to identify quotes with price relatives that fall outside the range of the main bulk of quotes.
These quotes can be identiﬁed from quote reports for the item, then investigated and appropriate action taken if necessary.

12.51 Price quote report.

This consists of a range of information on an item that the index dispersion report has highlighted as warranting further investigation.

Information listed may include current price, recent previous prices and base price, together with locations and types of shop.
The report can be used to identify the quotes that require further investigation and also to investigate rejected prices.

Algorithms 12.52 Algorithms can be created which may be used to identify and invalidate price movements that diﬀer signiﬁcantly from the norm for an item.

For some sea- sonal items for which price movements are erratic, it may be more appropriate to construct an algorithm to look at price level rather than price change.

12.53 An example is the Tukey algorithm.


This oper- ates as follows: The ratio of current price to previous valid price (the price relative) is calculated for each price.

(In the case of items tested by price level rather than price change, this stage is omitted.)

 For each item, the set of all such ratios is sorted into ascending order, and ratios of 1 (unchanged prices) are excluded.

(In the case of items tested by price level rather than price change, the prices themselves are sorted.)
The top and bottom 5 per cent of the list are removed (this 5 per cent is parameter 1).

 The ‘‘midmean’’ is the mean of what is left.

 The upper and lower ‘‘semi-midmeans’’ are the mid- means of all observations above or below the median.

 The upper (lower) Tukey limit is the midmean plus (minus) 2.5 times the diﬀerence between the midmean and the upper (lower) semi-midmean.

This ﬁgure of 2.5 represents parameters 2 and 3.


The upper and lower values can be set independently if desired but are currently set to be equal.

 If the upper limit is negative, it is set to zero.

(If price levels are used, the lower limit is set to zero.)

 Price relatives, or price levels, outside the Tukey limits are ﬂagged as unacceptable and requiring amendment or further investigation.

12.54 The Tukey algorithm has a number of advan- tageous characteristics.
In particular, it produces intui- tively reasonable results; is consistent from month to month; is robust in the presence of outliers (in other words, adding in one or two rogue observations does not aﬀect very much the limits set by the algorithm); and is robust as data volume changes (i.e.

limits calculated from a subset of the data do not vary much from those cal- culated on the full data set).
12.55 Whilst algorithms can be an eﬃcient way of highlighting problematical data, a word of caution should be expressed about using them.

Analysts will ORGANIZATION AND MANAGEMENT want to assure themselves that their use does not result in systematic bias in the index.

This is a matter that may also need to be taken into account in any editing rou- tines, although it is less likely to be problematical in the context of manual editing.
Producing and publishing the index 12.56 In regard to producing and publishing the index, there are a number of organizational models that could be adopted for eﬀective working.
Considerations to be taken into account in deciding on the appropriate organizational structure include: – the need for clarity of reporting lines; – the need for a clear division of responsibilities; – centralized or decentralized management of ﬁeldwork (see above discussion on local collection, and the outsourcing of ﬁeldwork, paragraphs 12.6 to 12.14); – production management versus technical develop- ment; – compatibility with corporate structures in the national statistical institute, for example, in relation to quality management, methodological research, and dissemi- nation.
12.57 In some cases, for instance where little in- house expertise in ﬁeldwork practice exists, it may be advantageous for ﬁeldwork to be conducted by a dif- ferent organization in either the public or private sector.

In these circumstances it is important that an eﬀective contractual relationship exists with regard to the data.

There should also be agreed delivery targets and per- formance measures to cover such things as data delivery timetables, response rates and levels of accuracy.
Con- sideration should also be given to the independent auditing of the contractor’s work on a sample basis.

Monthly compilation 12.58 The system used for the regular computation of the index must be suﬃciently ﬂexible to allow for changes in the kind of data obtained.
For example, local price collection for purposively sampled products from the branches of a large supermarket chain may be super- seded by centrally collected prices for a statistical sample drawn from complete sales data made available by the head oﬃce of the chain.

In these circumstances, a modular approach may be seen as an advantage.

12.59 Analytical computations provide comparisons between the published index, or one or more sub- indices, and what they would have been using diﬀerent methods or data.
They help to explain why the index has moved as it has and they allow methodological experi- mentation.
The following examples of such investiga- tions serve to make clear some of the computational capabilities and data that are required: – alternative aggregations of sub-indices; – the eﬀects of diﬀerent weights; the eﬀects of introdu- cing newly signiﬁcant product categories; and price- updating of weights; 221 CONSUMER PRICE INDEX MANUAL: THEORY AND PRACTICE – number and duration of missing observations; how a diﬀerent method of estimating them would aﬀect the index; – comparison of indices computed with various sub- samples of the data as a means of estimating variance; variances of price ratios; – computation of a standard reference index (one with no explicit quality adjustments) so that an implicit quality index is obtained; – numbers of sampled products; rates of forced replace- ments; and lengths of time products remain in the sample; – frequency distributions of quality adjustments.
12.60 To examine such matters, the database must contain not only prices but detailed descriptions of product replacements, explanatory remarks attached to observed prices, and so on.

Generally, it will be found that historical databases will be too large to be stored live on the system and therefore will need to be archived.

Detailed documentation relating to the archived material will need to be kept to guard against loss of vital infor- mation caused by changes in computing staﬀ or com- puters.
Consideration should also be given to appointing a data custodian with responsibility for all archived records.

Spreadsheets 12.61 Spreadsheets may be used for compiling sub- indices that require special procedures, or where data are collected centrally, or on an uncertain timetable or to a diﬀerent timetable from that of other data collection, but eﬀective control procedures need to be put in place.
Examples of types of prices for which the use of separate spreadsheets may be useful include: air fares, hotel accommodation, newspapers and car rental.

Such use of a spreadsheet has the advantage of additional ﬂexibility and scope for combining responsibility for data collection, data input and com- putation.
The compiler’s specialized knowledge about the markets or outlets where these prices may be observed, combined with analytical tools applied to the spreadsheet, will help the compiler to detect any irregularities in the data, facilitate investigation of whether these reﬂect reporting or input errors, and allow for rapid rectiﬁcation.
The ability to jump be- tween numerical data entry and a chart displaying, for example, current-month and previous-month entries, helps the rapid and simple detection of anomalies.

The same person can then follow this up with the data supplier.

12.62 As time passes, the resolution of problems that have arisen and adaptation to new circumstances will result in changes in the spreadsheet.
Unless quality management controls are put in place, there is a danger that the spreadsheet will be understandable only by the person responsible and that it will not be properly documented.

If so, two unfortunate consequences can arise: If that person is absent, retires or moves to another job, his or her successor will ﬁnd it very diﬃcult 222 to maintain the continuity and quality of the sub- index.
New procedures introduced to deal with new circum- stances may be inconsistent with procedures used for other sub-indices for which other people are respon- sible.

12.63 Good documentation and good communica- tion with colleagues will diminish these risks.

At a mini- mum, there should be an insistence that the spreadsheets and changes in them are made understandable by the provision of adequately explicative row and column headings or of notes attached to headings.
Furthermore, changes in procedures or formulae, rebasing and the application of new weights should always be introduced by moving computation over to a new sheet within the workbook, not by modifying the old sheet.

The new sheet and the old sheet will then exist side by side so that they can be compared.

12.64 Inadvertent changes may be prevented by using passwords to cells containing formulae and by locking cells containing input data once editing is com- pleted.
Passwords should be known only to a limited number of people with authority to edit the spreadsheets.

Regular back-up by copying the whole workbook to another disk is also essential.

Introducing changes 12.65 Various checks should be carried out when introducing changes.

These may include a comparison of the old and new basis using data from parallel running of collections (e.g.
when handing over to a new collection contractor) or re-estimating backwards – for example, when new base prices are being imputed for a complete range of goods or services.

Any anomalies can then be investigated further.

Disaster recovery 12.66 A consumer price index will arguably be the most important and highest-proﬁle statistic that a national statistical institute produces and can aﬀect the widest range of users.
There is often a legal obligation for the CPI to be published within a short time period after the end of the month to which the data refer.

For exam- ple, in the European Union, there is a legal requirement to publish within 30 days of the reference period the Harmonized Index of Consumer Prices (HICP), which uses the CPI data sets from member States (although the Eurostat timetable is for publication two weeks earlier than this).
Any delay in publication can have a sig- niﬁcant impact on subsequent months, threatening future publications.

Signiﬁcant delays could take months to catch up, in order to return to the existing tight pub- lication timetables.

It is critical, therefore, that national statistical institutes develop a robust and tested disaster recovery plan, however unlikely the need to implement it.
12.67 There are a number of possible causes of dis- aster: – failure of an external contractor to fulﬁl obligations to supply information; – failure of the computer system; – major natural disaster or other event (e.g.

terrorist activity) aﬀecting the operations centres or head oﬃce of the national statistical institute.
12.68 Where the collection is contracted out, one of the most important requirements of a disaster recovery plan is to recruit an alternative permanent service sup- plier as soon as possible.

It is probable that, on termi- nation of a contract with an external provider, the national statistical institute could arrange to have the services supplied by a third party, but only on a tem- porary basis prior to re-letting the contract through a competitive tendering process.
12.69 Additional money may need to be obtained for implementing a computing disaster recovery plan.

Con- sideration needs to be given as to whether outsourcing the disaster recovery plan to a company specializing in the provision of back-up support or maintaining an in- house capability is the best option.

This will, in part, depend on the number of sites and locations at which the national statistical institute operates.
If the organization has a number of sites, some distance apart but linked by modern communications infrastructure, then there is less likelihood that they will all be aﬀected by a natural dis- aster.

12.70 The managers of disaster recovery plans will also need to consider: – full speciﬁcation of accommodation and associated requirements (e.g.
personal computers, telephones) associated with each site; – allocation of speciﬁc oﬃcers to speciﬁc duties for the disaster recovery period and identiﬁcation of each individual’s training needs; – investigation of practicalities and associated expenses for matters such as access to shared drives and sys- tems, including communication and the quality man- agement systems, from other sites; – conﬁrmation of costs, arrangement of site visits and liaison with procurement units in negotiating con- tracts.
Quality management and quality management systems 12.71 Statistical oﬃces are faced with the continuous challenge of providing a wide range of outputs and services to meet user, i.e.

customer, needs.

Thus a key element of quality is customer focus and the eﬀective dissemination of relevant, accurate and timely statistics.

In addition, it can be argued that quality management should include eﬀective customer education on the use of such statistics.
In these terms, success can be measured by the achievement of a high level of satisfaction amongst well-informed users.

12.72 For the quality management of a CPI, it can be argued that the priority area is quality control of the production process itself.

For most national statistical institutes, quality control of production will be an area which represents a high risk, given the complexity of the process and the ﬁnancial implications of an error in the index.
ORGANIZATION AND MANAGEMENT 12.73 If the principles of organizing and managing the collection of data, and subsequent processing of information to produce a consumer price index, are to be adopted, then it is vital that a system is in place to ensure that the data obtained, the processes involved in achiev- ing the speciﬁed outputs, and the formulation of the policies and strategies that drive them are managed in an eﬀective, consistent manner.
The processes should, wherever possible, be open to veriﬁcation; and mech- anisms should be put in place to ensure that outputs meet requirements – in other words, customer satisfaction.

Taken together, these elements form the basis of a quality management system.

12.74 There are varying perceptions about the meaning of quality but an important common thread is the requirement to react to and serve users of the CPI and to ensure continuous improvement in that service.
Thus the implementation of an eﬀective quality man- agement system requires a high level of understanding of customer needs and the translation of this into a coherent statistical and quality framework.

Such a framework is also necessary for putting together criteria for judging success.
User needs can be canvassed either formally through negotiation of contractual obligations which may or may not be legally binding, or less formally through talking to customers on a one-to-one basis or through customer surveys.

12.75 In many countries, issues relating to the gov- ernance of the national statistical institute are set down in a ‘‘framework’’ or similar document.
This deﬁnes the functions and responsibilities of the national statistical institute, and generally guides and directs the work of the oﬃce.

For instance, an objective stated in the frame- work document ‘‘to improve the quality and relevance of service to customers – both in government and the wider user community’’ provides a powerful statement for determining workplans.
12.76 This recognition of the importance of quality can be further endorsed by a published vision of the national statistical institute as a key supplier of author- itative, timely and high-quality information.

Such a vision can be encapsulated by publishing objectives in an annual business plan.

These objectives can include improving quality and relevance, thereby increasing public conﬁdence in the integrity and validity of outputs.
12.77 Performance can be measured against a com- bination of a number of factors, including accuracy, timeliness, eﬃciency and relevance.

There are a number of practical examples and case studies of quality sys- tems, illustrating how diﬀerent models may be applied.

Quality management systems 12.78 Various standards of best-practice standards can be exploited to help organizations to improve quality management.

Some of these standards have the added advantage of being internationally recognized.
12.79 Total quality management.

Total quality man- agement (TQM) is more closely identiﬁed with a man- agement philosophy rather than a highly speciﬁed and structured system.
The characteristics associated with 223 CONSUMER PRICE INDEX MANUAL: THEORY AND PRACTICE TQM and an eﬀective culture of quality in an organiz- ation include: – clearly deﬁned organizational goals; – strong customer focus; – strategic quality planning; – process orientation; – employee empowerment; – information sharing; – continuous quality improvement.

12.80 Benchmarking.
12.80 Benchmarking.

Benchmarking is a process of comparing with others, and learning from them about what you do and how well you do it, with the aim of bringing about improvements.

12.81 There are already a number of benchmarking partnerships operating within national statistical insti- tutes, some speciﬁcally considering the CPI.

The Aus- tralian Bureau of Statistics has been particularly active in this area, and undertook an exercise in 1998–2000 in partnership with the United Kingdom.
Benchmarking projects have also been undertaken in New Zealand, the Scandinavian countries and the United States.

12.82 Areas that can be considered when bench- marking a CPI collection may include: – timelines, accuracy and coverage of collection; – beneﬁts of index methodologies for various items, e.g.

geometric mean as against average of relatives; – frequency of collection and publication; – cost of collection per unit of commodity, etc.
12.83 European Foundation for Quality Management Excellence Model.

The Excellence Model (1994) con- structed by the European Foundation for Quality Management (EFQM) is a diagnostic tool for self- assessment.

The model is widely used by governmental organizations across Europe to improve quality and performance.

It may be described as a tool that drives the philosophy of TQM.
12.84 The EFQM Excellence Model focuses on general business areas and assesses performance against two sets of criteria – the ﬁrst consists of ﬁve criteria covering what the business area does (the enablers: lea- dership; people; policy and strategy; partnership and resources; and process), and the second consists of four criteria on what the business area achieves (the results: people results; customer results; society results; and key performance results).
Evidence based on feedback from focus groups, questionnaires and personal interviews is used to score performance, and a resulting action plan for improvement is introduced which is then included in the business plan.
12.85 Underlying the EFQM Excellence Model is the realization that business excellence – measured through customer satisfaction – is achieved through eﬀective leadership which drives policy and strategy, allocates resources compatible with that policy, and man- ages employees in such a way as to enable them to manage the processes.
12.86 In the case of national statistical institutes, where some procedures are governed by statute or reg- ulation, the use of the EFQM Excellence Model enables 224 continuous improvement to be taken forward across a range of processes and functions.

To work eﬀectively, it needs the commitment of senior managers, who must be responsible for leading any self-assessment.
However, unlike ISO 9000, where assessment is carried out by qualiﬁed auditors often from outside the work area (see below), the EFQM Excellence Model relies on the input from all staﬀ.

12.87 ISO 9000.

The International Standard ISO 9000 is an international quality standard for manage- ment systems (ISO, 1994).

A quality system is a common- sense, well-documented business management system that is applicable to all business sectors.
It helps to ensure consistency and improvement of working practices, including the products and services produced.

12.88 The ISO standards were fully revised as ISO 9001 in November 2000 to match current philosophies of quality management and views regarding the struc- tures that need to be in place to ensure that continuous improvement is maintained (ISO, 2000).
12.89 The revised standards give users the oppor- tunity to add value to their activities and to improve their performance continually by focusing on the major processes within the organization.

They will result in a closer alignment of the quality management system with the needs of the way the organization runs its business activities.

By meeting the ISO 9000 standard, an organization will come more into line with TQM and the EFQM Excellence Model.
the organization and reﬂect Scope for greater use of quality management techniques 12.90 Both ISO 9000 and the EFQM Excellence Model have received a great deal of international recognition over recent years.

At the same time, the use of benchmarking networks has also grown in promi- nence.
It is therefore pertinent to ask whether more coordinated use should be made of these and other quality management techniques at a strategic level in ﬁelds of statistics where the focus is on international comparability.

This is particularly so with statistics that are compiled for treaty purposes, for example by member States of the European Union following detailed meth- odological guidelines laid down in law.
12.91 The arguments are ﬁvefold: It is paramount that such important non-optional statistics whose production and uses are enshrined in legislation have the full trust of users.

 The quality of international comparisons is dependent on the weakest link, thus good-quality statistics from one country may be of little value if not matched by statistics of equally good quality from other countries.
There is a potential for misleading analysis and con- clusions arising from diﬀerences in the application of standard methodology.

 Empowerment in ensuring the establishment of ade- quate control processes is reduced when production is delegated to member States.

 There is limited scope for centralized validation and quality management when production is decentralized.
Performance management, development and training 12.92 An eﬀective performance management system for individuals is just as important as applying such a system to management structure.

Performance man- agement can be seen as a continuous process designed to improve work outputs by focusing on what people actually achieve rather than the amount of eﬀort they put into the work.
It should provide the link between the objectives of the individuals, those of their team and those of the wider organization, so that workplans are coherent across the organization, and everybody knows what they are doing and why they are doing it.

The performance management system should pro- vide clear objectives for monitoring and evaluation, to enable feedback on performance and also to assist with the identiﬁcation of the development needs of individuals.
Performance management should be con- tinuous.

Training requirements 12.93 Eﬀective training will help motivate staﬀ and equip them to deliver a good-quality CPI.

At its simplest, training will give a background understanding of the nature and uses of the index and how it is compiled.

Training and development takes many diﬀerent forms and may include: – tutoring by the line manager or supervisor; – attending an induction course or reading a manual; – accompanying an experienced price collector.
12.94 A written training plan is useful in identifying training and development needs in relation to the or- ganization’s goals and targets.

It can also be used to identify the resources required to deliver the training to meet these needs, and to evaluate whether the train- ing has been delivered eﬀectively and objectives have been met.

Speciﬁc training for compilers and collectors 12.95 Further training will be required for speciﬁc skills, depending on the roles of the individuals and their jobs.
Training should continue beyond the induction stage to cover changed procedures, and retraining where performance is unsatisfactory.

12.96 Price collectors will need to be trained speci- ﬁcally in ﬁeld procedures, including relations with shopkeepers, the selection and deﬁnition of a valid price, special rules for certain individual items (including sea- sonal items), how to complete forms and, where appropriate, how to use hand-held computers.
Compi- lers of the index will need to be trained speciﬁcally on validation procedures and consistency checking, the calculation of centrally collected indices, weighting ORGANIZATION AND MANAGEMENT procedures and how to aggregate prices, as well as on the treatment of seasonal items and special procedures relating to some sections (e.g.

housing).

It may also be beneﬁcial to provide training in local or national trading or statistical regulations, economics, and commodity information.
12.97 Signiﬁcant beneﬁts may result from the inter- action between price collectors and index compilers.

Beneﬁts will also be gained from liaison between the national statistical institute and commodity experts from industry.

Such experts can advise on issues such as how to identify quality features on particular items, for example electrical goods, personal computers, or cloth- ing and footwear.
12.98 It may be beneﬁcial if statisticians from head- quarters are personally responsible for supervising price collection in the area where the head oﬃce is situated, so that they can have ﬁrst-hand experience of the problems involved.

This will put them in a position to provide assistance where diﬃculties arise.

Equally, it is a good idea to arrange for regular visits to headquarters by groups of collectors and their supervisors.

It is good for morale.


Price collectors will, arguably, do a better job if they feel that they belong to a team, if they can see that their work is appreciated and if they feel that their problems are understood.

Visits to headquarters will help convey that the accuracy and conscientiousness of their contribution is recognized as being crucial to the quality of the index.
Visits to head oﬃce by price col- lectors also will help the statisticians to keep in touch with conditions in the ﬁeld and, for example, to obtain more information about new goods and aspects of quality change.

12.99 Similarly, compilers of the index may wish to visit the ﬁeld occasionally and participate in or simply observe the price collection.
This will provide them with a better appreciation of the practical problems associated with price collection and a better feel for data (and in consequence for the quality of the index), together with the skills required to help with price collection in the event of an emergency.

Documentation 12.100 A manual and other documents such as desk instructions may serve for initial training.
Later on such documents should enable the collectors and compilers to remind themselves of all the relevant rules and procedures.

The documentation should be well organized and well indexed so that answers to problems can quickly be found.

12.101 Documentation should be checked by all concerned and updated regularly.

The pile of pieces of paper containing amendments should never grow large, but should be replaced by a new consolidated version.
One way of achieving this is to have a loose-leaf manual so that individual pages can be replaced when- ever necessary.

Another option is to keep an elec- tronic version that can be updated by nominated individuals.

the updating of documentation is done in a systematic and controlled way.

A variety of software is available to help the statistician to do this.
important that It is 225 CONSUMER PRICE INDEX MANUAL: THEORY AND PRACTICE 12.102 The beneﬁts of using standard electronic software for documentation are threefold: – more eﬃcient production of documentation, because the software helps with the initial compilation of in- formation and reduces the need to print and circulate paper copies; – better-informed staﬀ, because they have immediate electronic access to the latest documentation, includ- ing desk instructions, with a search facility by subject and author; – better quality control, because authors can readily amend and date-stamp updates and because access to non-authors is restricted to ‘‘read only’’.
Reviews 12.103 Training may be seen as an essential part of continuous quality improvement.

Staﬀ may be invited to operational reviews where all team members have the opportunity to raise concerns and, where appropriate, tackle speciﬁc issues through individual or group training.

226 PUBLICATION, DISSEMINATION AND USER RELATIONS 13 Introduction 13.1 The consumer price index (CPI) is one of the most important statistical series.
Where statistics are categorized according to their potential impact, the CPI and its variants are always in the ﬁrst rank.

It follows therefore that it must be published, and otherwise dis- seminated, according to the policies, codes of practice and standards set for such data.
13.2 The CPI should therefore be: – released as soon as possible; – made available to all users at the same time; – released according to pre-announced timetables; – released separately from ministerial comment; – made available in convenient form for users; – accompanied by methodological explanation; – backed up by professional statisticians and econo- mists who can answer questions and provide further information.
13.3 Above all, the CPI should meet the Funda- mental Principles of Oﬃcial Statistics (United Nations, lan- 1994).

These principles are published in several guages on the websites of the United Nations and the United Nations Economic Commission for Europe (UNECE).

They refer to dissemination and to all aspects of statistical work.

These and other standards are dis- cussed in this chapter.
Time series presentation of level and change 13.4 It is common, though not universal, to give greatest prominence to indices that show changes in aggregate prices between the month for which the most up-to-date data are available and the same month one year earlier.

It is also usual to compare this annual change with the annual change shown one month pre- viously.

The model presentation in Box 13.1 on page 230 provides an example of this.
It is also possible to focus on the latest one-month change or to give some emphasis to quarter-on-quarter changes.

13.5 The arguments for the choices shown in the example are as follows.

The 12-month comparison pro- vides an indication of price changes in a reasonably long time frame, by reference to periods which may otherwise be expected to be similar year to year.

Thus, seasonal factors are unlikely to be inﬂuential.
Also, price changes that are often decided centrally, such as those relating to the tariﬀs of utilities, and changes in indirect taxes (which have a direct impact on prices), are usually on an annual timetable and occur in the same month or months each year.

There may nevertheless be one-oﬀ changes that can have an inﬂuence on the index.

13.6 Some press releases may give prominence to the month-on-previous-month change, especially for some components of the CPI.
Such data have to be presented with care to avoid suggesting, for example, that a 2 per cent change in one month is similar to a 24 per cent change over a year.

13.7 It is also virtually universal to set a reference month (or longer period) in the past for which the price index is set at 100.

All subsequent months then have index numbers which are percentages of the reference month or period.

Indeed, it is that index which is used as the basic ﬁgure from which the other changes are cal- culated.
13.8 Indices are usually shown only to one decimal place, as are the other changes mentioned here, so ﬁgures have to be rounded.

Rounding in these circumstances can, however, give a false impression of comparative change and must thus be explained, especially where prices are changing relatively little.
13.9 Care has also to be taken to diﬀerentiate between percentage points in the basic monthly index (which usually has 100 per cent set several years earlier) and, for example, percentage changes between one month and the next.

If in one month the index is, for example, 200 and the following month it is 201, then the change can be described as one percentage point (above the period when the index was set at 100) or as half a percentage point (where the previous month is taken as 100 per cent).
Both are valid, but they are percentages of diﬀerent points in the past.

It is therefore important to specify which is the base point of reference.

13.10 The reference period which is set at 100 is often referred to as ‘‘the base period’’.

But it is often a relatively arbitrarily chosen date, changed every few years, and not necessarily related to any point in time when methodologies may have changed or when a new basket of goods and services was introduced.
The status of the reference period should be made clear in the method- ological explanation.

13.11 The CPI is, by deﬁnition, an index and there- fore not a level or a series of absolute changes in prices.

Nevertheless, in the process of presenting the CPI, average prices are calculated for categories of goods and services.
It is thus possible to publish some average prices for groups of goods or services, and also to show the upper and lower bands of the prices from which the averages have been calculated.

Some users of the index ﬁnd average price levels useful; these averages should therefore be made available to researchers who may want 227 CONSUMER PRICE INDEX MANUAL: THEORY AND PRACTICE them.
It has to be noted, however, that data on price levels may be less reliable than the price change indices for any given group of goods or services.

13.12 So far this chapter has referred only to the broadest aggregates, without reference to subgroups of prices or to variants of the CPI which may include or exclude certain items.
All of the foregoing refers to the most common form of CPI, which is usually intended to refer to the ‘‘average consumer’’ in a speciﬁc country and to include virtually all consumer prices in that country.

But it can equally refer to regions of a country or to subgroups (such as pensioners), or to related or alter- native measures of price change.

Related or alternative measures, and sub-aggregate indices, are discussed in paragraphs 13.24 to 13.37 below.
Seasonal adjustment and smoothing of the index 13.13 The treatment of seasonal products and the estimation of seasonal eﬀects are discussed in Chapter 22.

In the present chapter we discuss the dissemination of such adjusted or smoothed series.

13.14 Most series of economic statistics are shown seasonally adjusted, as well as unadjusted.

Consumer price indices are, however, not usually seasonally adjus- ted, although they sometimes are.
Seasonal factors, for any series, are usually frequently recalculated using the latest data, so seasonally adjusted series can be changed in retrospect, but unadjusted CPIs are not usually revised.

In comparing one month with the same month a year earlier, it is assumed that seasonal patterns are much the same from one year to the next.

There may be, however, exceptional months when the usual seasonal change is advanced or delayed.
Such exceptional circum- stances should be noted as one of the likely causes of a change in the CPI or in one of its components.

13.15 13.16 Changes over periods of less than a year are of course subject to seasonal factors and, in order to dif- ferentiate seasonal factors from other factors, it is nec- essary to make estimates of seasonal eﬀects and to note them as factors that have contributed to changes in the index.
13.17 Although the CPI itself is not usually season- ally adjusted, some variants of the CPI may be seasonally adjusted, perhaps because they are more subject to sea- sonality and because they can be revised in retrospect if necessary.

If such variants are seasonally adjusted, it is important to explain why.

Seasonal adjustment usually leads to a smoother series than the original unadjusted one.

There are also other ways of smoothing a monthly series, for example using three-month moving averages.
13.18 Statistical oﬃces do not usually smooth the CPI series in their published presentations.

Consumer price changes are not usually so erratic from month to month as to disguise price trends.

If there is an erratic change, the producers of the index can usually explain the reasons for it.
In any case, where any seasonally adjusted or smoothed series is published, it is important to publish the unadjusted as well as the adjusted series, so 228 that the eﬀect of the adjustment process is clear to users who may wish to know what has happened to prices, whether or not the changes can be put down to seasonal factors.

Similarly, a full explanation should be given for the reasons why a particular seasonal adjustment pro- cedure has been followed.
Analysis of contributions to change 13.19 The CPI is an aggregate of many diﬀerent goods and services whose prices are changing at diﬀerent rates, some of which may be going up while others are going down.

Many users of the index want to know which goods or services have contributed most to chan- ges in the index, and which prices may be out of step with general price trends.
13.20 The statisticians who calculate the index are well placed to provide analyses of the contributions to the price change, and to do so at the same time as the index is published.

Suﬃcient detail should be made available so that users can see for themselves what has happened to various groups of prices.
In addition, to assist journalists and others working under time con- straints, the statistician should indicate the goods or group of products whose changes in price are the main contributors to the aggregate CPI, and also goods whose changes in price are the most diﬀerent from the aggre- gate.

The statistics can be presented in the form of tables and charts so that the trends may be compared.
Simi- larly, statisticians should indicate any reasons for price changes which may not be immediately obvious but are nevertheless discernible from the published ﬁgures.

For example, if there has been a sharp price rise or fall one year earlier, then it will aﬀect the current year-on-year change, whatever happens to prices currently.
13.21 Analysis of contributions to change should also refer to any pre-announced price changes, or major changes since the last price-reporting date, which will aﬀect the outlook for the index over the following months.

Economic commentary and interpretation of the index 13.22 In undertaking an analysis such as that de- scribed above, statisticians must be objective so that users of the data may diﬀerentiate clearly between the ﬁgures themselves and the interpretation of them.
It is therefore essential to take care to avoid expressing any judgement of the impact of current policy on price changes or the possible implications of price changes for future policies.

Whether the ﬁgures should be seen as good news or bad news is for the users to decide for themselves.

The statistician’s role here is to make it as easy as possible for users to form their own judgements from the perspective of their own economic or political views.
13.23 There are several ways of avoiding any ap- parent or real lapses in objectivity in the analysis.

The ﬁrst, and perhaps the most important, is to publish the ﬁgures independently of any ministerial or other politi- cal comment.

Another is to be consistent in the way the analysis is presented.

That is to say, the data should be presented in much the same format every month (see paragraphs 13.38 to 13.41 below).
For example, tables and charts should cover the same periods every month, and use the same baselines.

Presentation of related or alternative measures Core inﬂation 13.24 For the purposes of economic analysis, it is desirable to construct measures of ‘‘core’’ or ‘‘under- lying’’ inﬂation which exclude movements in the inﬂation rate that are attributable to transient factors.

In other words, measures of core or underlying inﬂation seek to measure the persistent or generalized trend of inﬂation.
Central banks, for example, need to have measures of the general trend of inﬂation when setting monetary policy.

For this reason, economists and statisticians are in- creasingly interested in developing measures of ‘‘under- lying inﬂation’’.

13.25 Several methods can be used to derive a mea- sure of underlying inﬂation.

Most measures focus on reducing or eliminating the inﬂuence of exceptionally volatile prices, or of exceptionally large individual price changes.
The most traditional approach is to exclude par- ticular components of the CPI on a discretionary basis.

The items to be excluded would be based on the statis- tician’s knowledge of the volatility of particular items, depending on the economic conditions of the country.

Items commonly excluded under this approach are fresh meat, fruit and vegetables, and petroleum.

Many coun- tries also exclude imported goods, government charges, and government-controlled prices.
In some countries, a calculation is made to exclude the eﬀect of indirect taxes such as VAT.

Of course, care must be taken so as not to exclude so many items that the remainder becomes only a small and unrepresentative component of the total.

13.26 Other methods include smoothing techniques, for example annualizing three-month average inﬂation.

A more diﬃcult method is to exclude outliers, that is those items with the highest or lowest increases.
Alternative indices 13.27 An example of an alternative index is a ‘‘tax and prices index’’ in which income tax and sometimes social security payments are taken into account.

Such an index estimates how much a taxpayer’s gross income needs to change in order to maintain his or her spending power.

It combines changes in direct (income) tax with changes in consumer prices.

13.28 Another example is an index which reﬂects changes in prices excluding indirect taxes (such as sales taxes) and duties.
When compared to the CPI itself, such an index indicates the eﬀects on prices of changes in indirect (e.g.

sales) taxation.

13.29 Both of these examples involve allowing for taxes in one form or another.

They are more complex PUBLICATION, DISSEMINATION AND USER RELATIONS than the CPI itself, and do not have the intuitive attrac- tion of an index which aims at tracking the change in prices of a typical basket of consumer goods and services.
As such, they should be presented as interesting and enlightening constructs based on the core index.

It must be made clear that they are not replacements for, or superior to, the CPI itself.

13.30 A further example is the European Union’s Harmonized Indices of Consumer Prices (HICPs), which are used to compare and aggregate price movements across European Union economies.
The HICPs do not use a common basket of goods for all the countries in which they are calculated, because buying habits are diﬀerent from one country to another, but the con- cepts and methods are nevertheless harmonized in other ways.

No European Union member uses the HICPs as its national CPI, and therefore member countries also pro- duce and publish their own indices.
Although the HICPs are already used as an important indicator in the zone within Europe which uses the Euro as its underlying unit of currency, the HICPs are nevertheless relatively new, and are still under development.

This is a case where the presentation of an alternative index may raise serious questions about whether it may be superior to the national CPI.
It is therefore important to explain clearly the underlying concepts (which generally diﬀerentiate the HICPs from national CPIs) and to explain in some detail the reasons why the results are diﬀerent.

The HICPs were not calculated before 1996, and therefore do not enable price comparisons before that date.

The starting date should be indicated if it is not obvious in any presentation.
13.31 Another concept is the cost of living index (COLI), which is usually deﬁned as an index that indicates the changes in the costs associated not just with buying the same basket of goods, but with providing the same utility or usefulness to the consumer.

Countries do not usually attempt to calculate a COLI on a regular basis, but users frequently refer to the CPI as a cost of living index.

It should be made clear, in any background notes, whether this is indeed the concept underlying the CPI.
Sub-aggregate indices 13.32 Countries commonly calculate price indices for hundreds of products (for example bread or foot- wear), based on thousands of individual price records.

The number of possible sub-aggregates is therefore very large indeed.

13.33 One kind of sub-aggregation is the grouping of sets of items or products which, when the sets are taken together, comprise the whole of the CPI.

An important consideration here is the relationship of products within the subgroups.
For example, an index may be presented for food and, under the heading of food, indices may be presented for subgroups such as cereals and vegetables.

13.34 One of the ﬁrst considerations in present- ing such sub-aggregate data for related products is consistency.

That is to say, there should be a set of sub- aggregates for which indices are calculated and pre- sented each month.

Users commonly attach great importance to being able to continue their analysis from month to month.
229 CONSUMER PRICE INDEX MANUAL: THEORY AND PRACTICE 13.35 Another consideration is international stan- dardization of the division of the index into groups of goods and services, which enables comparison between countries.

Some countries also have their own sub- aggregate groupings which may predate the current inter- national standard.

The generally accepted international standard for the presentation of sub-aggregates is the Classiﬁcation of Individual Consumption according to Purpose (COICOP).
It is used, for example, in the HICPs.

Because COICOP deﬁnes groups of items by the general purpose for which they are used (e.g.

‘‘transport’’ or ‘‘housing and household services’’), it combines goods and services within the same subgroups.

Many national classiﬁcations are, however, composed of subgroups in which goods and services are never in the same subgroup.
Where the national CPI is sub-aggregated by divisions other than the international standard, it is advisable either to present a breakdown also by COICOP or at least to show how the national classiﬁcation compares to the international standard.

COICOP and the related Central Product Classiﬁcation (CPC) are discussed in more detail in Chapter 3 of this manual.

13.36 A further type of sub-aggregate index is an index which is essentially the same as the CPI except that it excludes certain items from it.
The core index discussed earlier is an example.

It could also be argued that the HICPs are such an index because they exclude certain non- 13.37 monetary expenditures.

Some countries publish, in addi- tion to their all-items CPI, an index or indices which exclude certain expenditures.

An example is an index which excludes mortgage interest payments from housing costs.

In the presentation of all related or alternative measures, their deﬁnitions should be made clear.
It is also advisable to give the reasons for their publication.

Most importantly, it should not be suggested that the sub-aggregate index is more meaningful than or superior to the CPI itself.

Press release, bulletin and methodological statement 13.38 The model presentation of a CPI in Box 13.1 is an example of the ﬁrst page of a press release for a ﬁc- titious country.

Other formats are possible.

For example, the presentation might include a seasonally adjusted index.
As indicated in the model, the presentation should contain the following information: – details of issuing oﬃce; – date and time of release; – percentage change in new month over the same month one year earlier; – comparison with change in previous month; Box 13.1 Model presentation of consumer price index Ofﬁce of [name of country] Statistics Friday 18 February 2000, for release at 11.00 a.m. CONSUMER PRICE INDEX (CPI) JANUARY 2000: PRESS RELEASE In January 2000, consumers were paying 1.0 per cent more than they did in January 1999 for the goods and services in the CPI basket.
This 12-month change was lower than the 12-month change recorded in December (1.5 per cent) but higher than in November (0.9 per cent).
Percentage change in the consumer price index over the same month of the previous year, for the last ﬁve years 1995 1996 1997 Year 1998 1999 2000 Main contributions to the overall 1.0 per cent increase The largest increase was in the prices of clothing and footwear, with smaller increases in recreation and culture.

Within the energy group of prices, there was a signiﬁcant increase in gas tariffs.
There were falls in the prices of furnishings and household goods.

The changes in product groups are shown in the table on page x of this release.

Issued by the Ofﬁce of Xxxxx Statistics, address xxxxxx Press enquiries 1 111 1111; Public enquiries 2 222 2222 (name of a contact is helpful) Background notes on the CPI are given in the annex to this note.
More notes and more details are given in our Internet site at XXX 230 – information on the product groups which contributed to the change and on any signiﬁcant component price; – reference to where more information can be found.

Note that no judgements are oﬀered on policy or eco- nomic reasons for the price change, and no judgement is given on whether the change is good or bad.

13.39 What is not obvious from just one example is that the format of the press release should be the same from month to month.
Using a consistent format is important in order to avoid appearing to choose a dif- ferent format to indicate a preferred trend, for example from a selected starting date.

13.41 13.40 Other pages of the press release should give the monthly indices (base period equals 100) from which the percentage changes are calculated.

Similar indices should also be given for major groups of goods and services.
Charts may also be used to illustrate, for example, which prices have contributed most or least to the overall CPI.

If any other consumer price variant is also being published, then the diﬀerences between the indices should be brieﬂy explained, including any methodolo- gical diﬀerences.
Such variants that require explanation include, for example, a national index based on the European Union’s HICPs methodology, any regional indices, or versions of the CPI that exclude particular components of consumer expenditure such as house purchase.

The press release should include a short note on methodology, similar to that given in Box 13.2.

More detailed explanation could be given in a handbook.
International standards concerning the dissemination of consumer price indices 13.42 There are many international standards which apply, in general terms or speciﬁcally, to the CPI.

The introduction to this chapter lists some of the broad principles which are reﬂected in many of the interna- tional standards in some form.

One very general stan- dard, but by its nature a fundamental one, is the United Nations Fundamental Principles of Oﬃcial Statistics.
It is available on the web sites of the UNECE and the United Nations in several languages.

It refers not just to dis- semination but to all aspects of statistical work.

13.43 The International Monetary Fund (IMF) standards are particularly pertinent in regard to dis- semination.

There are two which refer to statistics includ- ing consumer price indices.

One is the General Data Dissemination System (GDDS), and the other is the Special Data Dissemination Standard (SDDS).
The GDDS provides a general framework, with some speciﬁc indicators deﬁned as ‘‘core’’ and others deﬁned as ‘‘encouraged’’.

The SDDS is based on the GDDS frame- work, but is more demanding and applies only to those countries that choose to subscribe to it in writing to the IMF Board.

Both standards are available on the IMF web site.


13.44 Under the heading of quality, the GDDS refers to the necessity to provide information on sources and methods, as well as on component details and PUBLICATION, DISSEMINATION AND USER RELATIONS Box 13.2 Model note on methodology – to be included in press releases on consumer price indices What is the consumer price index (CPI) measuring and how is it done?

The all-items consumer price index (CPI) is the main measure of what is commonly called inﬂation.
It measures the change in prices, on average, from month to month, of the goods and ser- vices bought by most households.

Prices are collected each month from shops and other sup- pliers of goods and services.

The pattern of household expen- diture on these goods and services is derived from a regular household budget (or expenditure) survey.

The prices and spend- ing patterns are then combined to calculate the price indices for groups of goods and services and for the all-items index.
The overall index, with all of its component indices, is published each month in our CPI Bulletin.

The Bulletin also contains more information on the methodology used in calculating the CPI.

A small booklet is also available.

For a detailed account of the methodology used in calculating the CPI, please see the CPI technical manual.
For more information on these publications, and how they may be obtained, please refer to our web site at www:ous:gov or telephone the numbers given on the front of this press release.

checking procedures.

Under the heading ‘‘integrity’’, it refers to declared standards of conﬁdentiality, internal government access before data release, identiﬁcation of ministerial commentary, and information on revision and advance notice of changes in methodology.
Under the heading ‘‘access by the public’’, it refers to the need for pre-announced release dates and simultane- ous access for all users.

In the tables of data categories, it refers to the CPI as a core indicator which should be issued monthly, within one to two months of the data collection date.

All of these standards are reﬂected in the present manual.

The ILO has also published guidelines concerning dissemination practices for labour statistics (ILO, 1998), which are available on the ILO web site.
Timing of dissemination of the consumer price index 13.45 The CPI should be released as soon as possible, but it is equally important to release the index according to a strict timetable.

It is also important to publish the timetable of release dates as far in advance as possible.

Having a ﬁxed release date, published well in advance, is important for two main reasons.

First, it reduces the scope for the manipulation of the release date for political expediency.
Second, it gives conﬁdence to users that the release date is as soon as possible and has not been delayed (or brought forward) for purely political reasons.

A third advantage is that users know when to expect the ﬁgures and can be prepared to use them.

Timeliness of release versus data accuracy 13.46 The IMF’s GDDS, discussed in paragraphs 13.43 and 13.44 above, recommends that the CPI be 231 CONSUMER PRICE INDEX MANUAL: THEORY AND PRACTICE released within one to two months of data collection each month.
It is usual, in practice, for most countries to release the CPI in the middle of the month after the month to which the index refers.

This is possible because, in many cases, the data are collected mainly over a lim- ited period in the middle of the month to which the latest data refer.

Thus the statisticians have some time to check and analyse the data, and to prepare the many tables and charts in which the data will be disseminated.
13.47 The accuracy of the index is particularly impor- tant because so much depends on the CPI.

In addition to the economic policy implications of the index, the CPI is used in most countries in a variety of contracts.

Perhaps the best-known contractual use is the indexing of wages and salaries.

Also, partly because it is rare for more data to emerge after the CPI is published, and partly because of the way in which the index is used in contracts, it is very rarely revised.
This represents a major diﬀerence between the CPI and other economic or socioeconomic aggregates.

It follows that, although timeliness is impor- tant, the timetable must allow time for the data to be properly prepared and thoroughly checked.

After the release date, in most cases, a revision to the non- seasonally adjusted CPI would not be permissible.

The HICPs of the European Union are an exception and are revised from time to time.
If any series is revised, then of course the changes must be fully described and explained when the new data are released.

If there is any method- ological change, this is usually known in advance.

Users should be warned before any such change occurs.

13.48 Access to data 13.49 With the CPI as with other statistics, users should be allowed access to as much data as possible for two main reasons.

First, some users ﬁnd the detailed data very useful in their analysis.
Second, access to the data inspires conﬁdence in the data.

13.51 13.50 There are, however, limits on the quantity of data that can be made available to users.

One reason is conﬁdentiality, which is addressed in the next section of this chapter.

Another is the quantity of data that most users can absorb.

A further reason is the cost of pub- lishing large quantities of data which few users may need.
In general, the CPI and its major components are deemed to be of such wide importance that they are made available free through press releases.

More detailed data are, however, often published only in books and other media, and are charged for in order to recover some of the dissemination costs.

Similarly, special ana- lyses made at the request of particular users are usually charged for at a rate commensurate with the work involved.
13.52 The quantity of data to which users should be given access through the various possible media is also discussed in paragraphs 13.53 to 13.58 below.

Conﬁdentiality 13.53 Although, in general, as much data as possible should be made available to users, there are reasons why 232 conﬁdentiality is important in some instances.
First, some data are supplied by retailers and others on the understanding that the data will be used only for the purpose of aggregation with other data and will not be released in any other form.

This can be especially impor- tant where the data are given voluntarily, as they often are.

Second, only a sample of particular brands is priced as representative of a much larger group of products.
If it is known which brands are included in the index and which are not, then it might be possible to bias compo- nents of the index by manipulating a small number of prices.

13.54 Even the knowledge that price data are, or might be, collected on one particular day in the month could enable some component price indices to be biased by retailers or others choosing to change prices on a particular day.

This is, however, only a short-run danger and cannot be sustained.
Electronic dissemination 13.55 The World Wide Web has several advantages as a dissemination medium.

For the data producer, dis- tribution costs are relatively small.

No printing or mailing costs are involved.

As soon as the data are on the Web, they are available to all Web users at the same time.

Putting a large amount of data on the Web costs little more than putting on a smaller amount.
Web users can download the data without re-keying, thus increasing speed and reducing transmission or transposition errors.

13.56 Among the disadvantages of dissemination via the World Wide Web is that not all data users have equal access to the Web.

Another important disadvantage is that users may go straight to the data, without read- ing the metadata which may be crucial to the proper understanding of the data.
Also, it may be as easy for a user to disseminate the CPI widely by electronic means as it is for the statistical oﬃce, thus enabling users to pre- empt the producers by circulating the index in advance of the release time, perhaps without the metadata which may be essential to a proper understanding of the ﬁgures.

Ideally the CPI, complete with any essential metadata, should be released simultaneously to the press and other users.
One way in which some statistical oﬃces are ensuring this is to bring the journalists together perhaps half an hour before the oﬃcial release time, provide them with the printed press release, explain the data and answer any questions.

Then, at release time, the journalists are permitted to transmit the ﬁgures to their oﬃces for wider distribution.

13.57 13.58 In essence, care must be taken to ensure that the CPI is available at the same time to all users, regardless of the dissemination medium used.
User consultation Different uses of consumer price indices 13.59 The diﬀerent uses of CPIs are discussed in some detail in Chapter 2.

It is important to explain to potential users of the CPI which are suitable uses and which are not.

To this end, it is important to explain how the CPI is constructed, and to provide details of its sources and methods.
It is also important to make readily available explanations of alternative indices or sub- indices, indicating how their uses diﬀer from the uses of the CPI itself.

Presentation of methodology 13.60 When the CPI is published each month, users are anxious to see the main ﬁgures and to use them.

Users do not generally want to be burdened with explanations of the methodology underlying the data.
Nevertheless, methodological explanations must be accessible to those who may want them, and in forms which are compre- hensible to users with diﬀerent levels of expertise and interest.

Any signiﬁcant changes in methodology must be fully explained, and notiﬁed as far in advance as possible of the change being made.

13.61 In addition to a brief statement in press releases (see paragraphs 13.38 to 13.41 above), methodological explanations should be available on at least two levels.
Non-experts should be able to refer to a booklet which explains the history, principles and practice underlying the CPI and any alternative measures which may also be available.

A more thorough explanation of sources and methods should also be readily available for those users who are suﬃciently interested and, for example, for statisticians who may be working on the production of the CPI for the ﬁrst time.
The information must also be kept up to date despite the pressures to devote time to the output at the expense of documentation.

As noted else- where, the ready availability of a full explanation of sources and methods is essential to conﬁdence and trust in the CPI.

Role of advisory committees 13.62 For a statistical series as important as the CPI, it is essential for there to be an advisory committee, or set of committees, representing users and producers.
There are many contentious issues in the construction of the CPI.

In many countries there have been ﬁerce arguments about, for example, which components should be inclu- ded and excluded.

The role of an advisory committee is to consider and to advise on contentious and other PUBLICATION, DISSEMINATION AND USER RELATIONS issues.
Perhaps an equally important role of an advisory committee is that its very existence provides reassurance that the CPI can be trusted and is not a tool of govern- ment propaganda.

13.63 In those countries where advisory committees have not been the norm, there may be a fear on the part of statisticians that including non-governmental participants may raise expectations beyond what the statisticians can deliver, thereby increasing dissatisfac- tion among the general public.
In fact, the inclusion of non-governmental users can lead to a greater under- standing of the realities and the practical constraints to meeting theoretical needs.

This is the usual experience of oﬃces that already have advisory bodies which include representatives of all the major constituencies, both inside and outside government.
It is therefore important that the advisory committee should comprise people such as academics, employers, trade union rep- resentatives and others who have an interest in the index from diﬀering points of view.

It is also important that the reports of the advisory committee are made available to the public fully and without undue delay.

Explaining index quality 13.64 The CPI is regarded with suspicion at many diﬀerent levels.
It usually refers to the average consumer, but each consumer has a diﬀerent spending pattern from the spending patterns of others and may notice changes in one set of prices but not in others.

More importantly, perhaps, there is criticism of the index because of suspi- cion that it does not keep track of newer types of goods and services, changes in the quality of products, or newer types of retailing.
13.65 In the light of such suspicion, it is impor- tant for the producers of the index to be willing to dis- cuss these issues and to explain how they are being dealt with.

As with other issues discussed here, the producers of the index must be open about their methods and the extent to which they can, or cannot, overcome the potential or real problems which have been identiﬁed.
It follows that the statisticians who produce the index should publish explanations of quality aspects, whe- ther or not the quality of the index is currently being questioned.

233 THE SYSTEM OF PRICE STATISTICS 14 Introduction 14.1 This chapter focuses on the value aggregates for goods and services that relate the major price indices, including the consumer price index (CPI), to one another.
The chapter provides a deeper context for the domain of the CPI covered in Chapter 3 and the index weights dealt with in Chapter 4.

It also deepens the context for deﬁning the sample unit and the set of products, discussed in Chapter 5.

14.2 We begin by deﬁning a value aggregate for a domain of goods and services as the sum of the products of the prices and quantities of those goods and services.
A price index may be characterized as the factor giving the relative change in this value aggregate arising from changes in prices.

As such, all the major price index formulae can be expressed as weighted averages of price relatives whose weights are the shares of items in the value aggregate.

For the best-known price index for- mulae expressed as value aggregates of share-weighted averages of price relatives, see Chapter 1, equation (1.2) and Chapter 15, equation (15.8) for the Laspeyres index.
See Chapter 1, equation (1.3) and Chapter 15, equation (15.9) for the Paasche index, and Chapter 1, equations (1.11)–(1.12) and Chapter 15, equations (15.21) and (15.81) for the Walsh and To¨ rnqvist indices.

As the geometric mean of the Laspeyres and Paasche indices, index of Chapter 1, equation (1.10) the Fisher ideal and Chapter 15, equation (15.12) is also a function of expenditure shares derived directly from the value aggregate.
14.3 To deﬁne a price index, we ﬁrst need to know several things about the value aggregate.

The value aggregate deﬁnes the following aspects of a price index: which commodities or items to include in the index; how to determine the item prices; which transactions that involve these items to include in the index; how to determine the weights, and from which sources these weights should be drawn.
Besides the content of the value aggregates for the major price indices, we also discuss in this chapter their valuation and timing properties.

These properties bear importantly on how compilers deﬁne the prices and weights of price indices.

14.4 The four principal price indices in the system of economic statistics are the consumer price index (CPI), the producer price index (PPI), and the export and import price indices (XPI and MPI).
They are well- known and closely watched indicators of macroeco- nomic performance.

They are direct indicators of the these indices are important purchasing power of money in various types of trans- actions and other ﬂows involving goods and services.

Consequently, tools in the design and conduct of the monetary and ﬁscal policy of the government.

They also are used as deﬂators to provide summary measures of the volume of goods and services produced and consumed.
They thus are also used to inform economic decisions throughout the pri- vate sector.

They do not, or should not, comprise merely a collection of unrelated price indicators, but pro- vide instead an integrated and consistent view of price developments pertaining to production, consumption, and international transactions in goods and services.

By implication, the meaningfulness of all of these indices derives in no small measure from the meaningfulness of the value aggregates to which each refers.
Although there are other important price indices, most of which also are discussed in this chapter, these four constitute the backbone of the system of price statistics in most coun- tries, and they will be given special attention.

14.5 Paragraphs 14.8 onwards establish the rela- tionships among the four major price series by asso- ciating them with certain of the interlocking aggregates deﬁned in the System of National Accounts 1993 (SNA 1993).
The system of national accounts (SNA) has gone through various versions over the years, the 1993 edition of this manual being the latest.

We will use SNA to refer to the system of national accounts generically, and SNA 1993 to refer speciﬁcally to the most recent version, as appropriate.

The CPI draws its coverage from a variety of accounts in the SNA.
At various points along the way, we note whether and how the composition of each value aggregate in the national accounts relates to the aggre- gate on which the CPI may be deﬁned.

Besides the four main price indices and an array of additional useful price indices, we brieﬂy consider labour compensation indices and purchasing power parities in the system of economic statistics.
14.6 As noted in Chapter 2, the CPI is constructed for a range of uses in various countries, but we can identify two broad themes: the consumption (sometimes called the cost of living) CPI, and the transactions (often called the inﬂation) CPI.
Advocates for the transactions CPI often refer to it as an acquisitions CPI, following the language of the earlier Consumer price indices: An ILO manual (Turvey et al., 1989) which used this terminology to distinguish alternative treatments of, for example, owner-occupied housing (p. 15).
The term ‘‘acquisitions CPI’’ has a diﬀerent meaning in the SNA, referring to households’ consumption of goods and services secured not only by themselves, but also by non-proﬁt institu- tions and government on their behalf.

We thus use the 235 CONSUMER PRICE INDEX MANUAL: THEORY AND PRACTICE term ‘‘transactions’’ instead.

In the ILO manual’s ter- minology, what we call a consumption CPI would have been called a ‘‘uses’’ CPI.

Either is consistent with cur- rent SNA terminology.
14.7 Both types of CPI are oriented towards the price experience of households, but, as its name implies, the consumption CPI focuses on the prices of items on which households make ﬁnal consumption expendi- tures, while the transactions CPI focuses on the prices of items on which households make monetary ﬁnal expenditures on consumption and capital formation.
Consumption CPIs thus exclude capital formation expenditures by households (for example, on their own dwellings), but may include both monetary and imputed consumption expenditures (for example, the imputed rent paid by homeowners on their own dwell- ings).
Transactions CPIs focus only on the prices of items on which households make monetary ﬁnal expenditures, and thus may include household capital formation expenditures (for example, net acquisitions of dwellings), but categorically exclude expenditures that must be imputed in order to cover households’ eﬀective consumption of goods and services.
In this chapter, we will further explain the concepts of institutional sector and type of transaction from the SNA that deﬁne the distinction and the relationship between the consump- tion and transactions CPI.
In each of the following sections, as relevant, we will discuss the kinds of expenditures deﬁning the items and weights appropriate for each of these two main types, referring to the sum of expenditures corresponding to the consumption CPI as expenditure aggregate #1 and that for the transactions CPI as expenditure aggregate #2.
National accounts as a framework for the system of price statistics 14.8 The system of national accounts is the core system of value aggregates for transactions and other ﬂows in goods and services.

It is clearly of broad eco- nomic interest.

Granted, the value aggregates of the major price indices need not be coincident with the major value aggregates in the national accounts.
The national accounts aggregates, however, represent the major ﬂows of goods and services and levels of tangible and intan- gible stocks in the economy.

The major price indices therefore should have a clear relationship to these ag- gregates.

This chapter explains the value aggregates now in common use by national authorities for the major price indices, or planned for future use, by assembling them from components identiﬁed in the SNA.
14.9 The SNA 1993 describes the system of national accounts as follows: 1.1 The System of National Accounts (SNA) consists of a coherent, consistent and integrated set of macro- economic accounts, balance sheets and tables based on a set of internationally agreed concepts, deﬁnitions, classi- ﬁcations and accounting rules.

It provides a comprehen- sive framework within which economic data can be compiled and presented for purposes of economic analy- sis, decision taking and policy making.
236 The accounts cover the major economic activities taking place within an economy, such as production, con- sumption, ﬁnancing and the accumulation of capital goods.

Some of the ﬂows involved, such as income, sav- ing, lending and borrowing, do not relate to goods and services and do not factor into price and quantity com- ponents.
However, the SNA also contains a compre- hensive framework, the supply and use table, discussed in more detail below, which establishes and displays the interrelationships between all the main ﬂows of goods and services in the economy.

The coverage and contents of these ﬂows are deﬁned, classiﬁed and measured in a conceptually consistent manner.
Within this table, the linkages between major ﬂows of goods and services associated with activities such as production, consump- tion, distribution, importing and exporting can be seen in a simple and direct way.

The table provides an ideal framework for designing and organizing a system of internally consistent price statistics that relate to a set of economically interdependent ﬂows of goods and services.
The table not only establishes the interrelationships be- tween consumer, producer, import and export prices themselves, but also their linkages with price indices for major macroeconomic aggregates such as gross domestic product (GDP).

14.10 In this overview of price indices, we ﬁrst take a top-level view of the major national accounts aggregates.
We then begin a review of the underlying construction of these aggregates by considering ﬁrst the types of eco- nomic agents in the economy that are recognized in the national accounting system, and second, the economic accounts kept on them involving goods and services ﬂows that build up to the main aggregates.
As these accounts are built up from their foundations, precise relationships emerge between the well-known headline price indicators – the PPI, CPI, XPI and MPI – and the closely watched national accounts aggregates.

Aggregate supply and use of goods and services 14.11 At the most aggregate level, the supply and use of goods and services in the national accounts is the simple textbook macroeconomic identity equating total supply with total uses.
Total supply is the sum of output Y, imports M, and taxes less subsidies on products T. Total use is the sum of intermediate consumption Z, the ﬁnal consumption of households C and government G, capital formation I, and exports X: Y+M+T=Z+C+G+I+X (14.1) 14.12 Rearranging this identity by subtracting inter- mediate consumption and imports from both sides, we arrive at the familiar alternative expressions for GDP from the production (value added) and expenditure approaches: (Y Z)+T=Value added+T C+C+I+X M =Gross domestic product (14.2) internationally recognized as the GDP is, of course, central national accounts aggregate for measuring economic performance.
It is essentially a measure of production, as distinct from ﬁnal demand.

More precisely, it measures the value added of the productive activities carried out by all the economic agents resident in an eco- nomy.

As imports are not included in GDP, a price index for GDP tracks internally generated inﬂation.
Compiling indices for tracking the parts of relative change in GDP and its components that can be attributed to price and volume change is among the most important objectives for the development of price statistics in modern statistical systems.

14.13 As explained in more detail later, the supply and use table in the SNA is a comprehensive matrix covering the economy as a whole that exploits the iden- tities (14.1) and (14.2) at a disaggregated level.
Each row of the matrix shows the total uses of a commodity, or group of commodities, while each column shows the total supplies from domestic industries and imports.

The table provides an accounting framework that imposes the discipline of both conceptual and numerical con- sistency on data on ﬂows of goods and services drawn from diﬀerent sources.

The ﬂows have to be deﬁned, classiﬁed and valued in the same way, while any errors have to be reconciled.
The table provides a good basis for compiling a set of interdependent price and quantity indices.

In the following sections, we consider the various elements or building blocks that make up the table before examining the table as a whole.
Institutional units and establishments 14.14 In building the accounting system and the major aggregates Y, M, T, Z, C, G, I and X of equations (14.1) and (14.2), the SNA 1993 ﬁrst organizes the econ- omy of a country into the kinds of entities or agents that undertake economic activity.

These agents are called institutional units and comprise ﬁve types resident in the economy, as well as a single non-resident category, the rest of the world.
An institutional unit is said to be resident in an economy if its primary centre of economic interest is located there.

A centre of economic interest is operationally deﬁned in part by the duration of physical presence.

For example, a household is resident in an eco- nomic territory if it lives within the territory’s boundaries for a year or more.
The ﬁve types of resident institutional units are: non-ﬁnancial corporations; ﬁnancial corpora- tions; general government; households; and non-proﬁt institutions serving households (NPISHs).

The SNA 1993 associates with institutional units the ability to hold title to productive assets, and thus they represent the smallest units on which complete balance sheets can be compiled.
14.15 As noted earlier, institutional units can engage in producing and consuming goods and services and in capital formation, accumulating goods and services as productive tangible and intangible assets.

To analyse production, the SNA 1993 identiﬁes a smaller unit or agent than an institutional unit, called an establishment or local kind of activity unit (LKAU).
Within an insti- tutional unit, the establishment is the smallest unit organized for production whose costs and output can be separately identiﬁed.

Generally, establishments special- THE SYSTEM OF PRICE STATISTICS ize in the production of only a few types of output at a single geographical location.

To compile productivity statistics, analysts also need detail on produced and non- produced non-ﬁnancial assets (capital) by establishment from multi-establishment institutional units.
This is because, as we will see, these statistics use an industry or activity classiﬁcation of establishments rather than insti- tutional units.

Some institutional units may own estab- lishments in more than one industry.

On the other hand, an account of ﬁnancial assets and liabilities by estab- lishment is not needed and not generally available from the accounts of institutional units owning multiple establishments.

The latter would be necessary to make establishment balance sheets.
14.16 The SNA 1993 classiﬁcation of institutional units into sectors is shown in Box 14.1.

The SNA 1993 classiﬁcation of institutional units does not strictly follow the legal status of institutional units, but rather their function.
Hence, a government-owned non-ﬁnan- cial enterprise producing output sold at prices sub- stantially covering its costs and for which a balance sheet can be compiled would be classiﬁed as a non-ﬁnancial corporation, along with non-ﬁnancial corporations that are corporate legal entities.

For further details, see SNA 1993, Chapter IV.

Notice that the SNA 1993 institutional sectors represent the units typically covered in economic and household censuses and surveys.
The SNA focuses on the activities of institutional units that are resident in a nation or economic territory.

It makes provision for the rest of the world (S.2 in Box 14.1) only to capture the transactions of resident institutional units with non- residents.

Transactions of non-residents with other non- residents are out of scope for the national or regional accounts of a given country or region.
14.17 The classiﬁcation of household institutional units into sectors is highly relevant for analysing the incidence of price change.

As shown in Box 14.1, the SNA 1993 deﬁnes household subsectors according to the major source of income: mixed income (mostly proﬁts of household enterprises), compensation (wages, salaries and compensation in kind), or property income (rents, dividends and interest).

There are not the only sectors of households that may be of interest to users of the CPI, however.
In addition to the source of income, analysts often (perhaps more often) are inter- ested in the level of income.

The shares of particular goods and services in household expenditures are likely to show more variation across income level than across major source of income.
For example, to shed light on the price experience of poor (low-income) households we would want to know whether there is a signiﬁcant diﬀerence in the shares of expenditure on speciﬁc goods and services for poor as compared with non-poor households.

A good example would be the relative importance of expenditures for used durable goods.

As we will see, consumer durables are measured in the SNA on an acquisitions-less-disposals basis.
While poor households normally would be net purchasers of such goods, richer households would tend to be net sellers.

A change in the prices of used goods thus would have a very diﬀerent impact on the CPIs for the two groups of households.
237 CONSUMER PRICE INDEX MANUAL: THEORY AND PRACTICE Box 14.1 Institutional sectors in the System of National Accounts 1993 S.1 Total economy S.11 Non-ﬁnancial corporations Ultimate subdivisions: public, national private and foreign controlled S.12 Financial corporations Ultimate subdivisions: public, national private and foreign controlled S.121 Central bank S.122 Other depository corporations S.1221 Deposit money corporations S.1222 Other depository corporations, except deposit money corporations S.123 Other ﬁnancial intermediaries, except insurance corporations and pension funds S.124 Financial auxiliaries S.125 Insurance corporations and pension funds S.13 General government Alternate scheme n = 1, social security funds shown as a separate branch of government S.1314 Alternate scheme n = 2, social security funds included as components of central, state, and local branches, and S.1314 deleted S.13n1 Central government S.13n2 State government S.13n3 Local government S.1314 Social security funds S.14 Households Classiﬁed according to the largest source of income received S.141 Employers (mixed income, owning an unincorporated enterprise with paid employees) S.142 Own account workers (mixed income,1 owning an unincorporated enterprise without paid employees) S.143 Employees (compensation of employees) 2 S.144 Recipients of property and transfer income3 S.1441 Recipients of property income S.1442 Recipients of pensions S.1443 Recipients of other transfers S.15 Non-proﬁt institutions serving households (NPISHs) S.2 Rest of the world 1To understand how subsectors S.141 and S.142 of households are formed, an explanation of the term ‘‘mixed income’’ is in order.
This, in turn, requires consideration of the national accounts income concept of operating surplus.

The operating surplus of an enterprise is the residual of the value of output less purchases of goods and services, inputs, wages and salaries, employers’ social contributions (social security and pension payments), and taxes net of subsidies payable on production that are unrelated to products.
The mixed income of household unincorporated enterprises is algebraically deﬁned identically with the operating surplus of other enterprises.

However, for unincorporated household enterprises, the compensation of the owners or proprietors of the enterprise may not be included in the recorded compensation of employees item, and thus the diﬀerence between output and operating cost will include compensation for the owners’ labour.
The distinct terminology merely recognizes that the owners’ wages are often inextricably mixed with the operating surplus for these units.

2Compensation of employees comprises wages and salaries, and employer-provided beneﬁts comprising employers’ social con- tributions.

3Property income comprises interest, dividends and rent.
Accounts of institutional units 14.18 In equations (14.1) and (14.2), we identiﬁed the basic aggregates comprising the total supply and use of goods and services in the economy, and derived GDP in terms of these aggregates.

To see how to separate the price and volume components of supply and use, it is necessary to build these basic aggregates up from the institutional sector accounts of the economy’s economic agents.
In this process it is important to detail the pro- duction and consumption activities of these agents, as well as the types of goods and services they produce and consume.

The framework organizing this information is the supply and use table.

As this table is built up, we eﬀectively also begin to accumulate data on the product share weights s needed for computing price index for- mulae (Chapters 1, 3, and 15–17).
The basic accounts of the SNA in which all of these aggregates are recorded at the level of institutional units are the production, use of income, capital, and external goods and services accounts.
These accounts organize the information for the following top-level aggregates: Production account: output Y, intermediate con- sumption Z, and value added Y – Z; 238 Use of income account: household consumption C and government consumption G; Capital account: capital formation I; External goods and services account: exports X and imports M. Recording transactions in goods and services 14.19 Before turning to further elaboration on these four goods and services accounts, it is important to specify how each entry in the value aggregates compris- ing them is to be recorded.
The items i in the value ag- gregate equation (15.1) of Chapter 15 represent detailed goods and services ﬂows classiﬁed into categories of transactions.

There are two deﬁning aspects of recording transactions: timing and valuation.

14.20 Regarding the timing of transactions, to asso- ciate each transaction with a date, the national accounts consider a transaction to have been consummated when a liability to pay is created between the units involved.
For ﬂows of goods and services, this occurs when the ownership of the good is exchanged or when the service is delivered.

When change of ownership occurs or the service is delivered, a transaction is said to have accrued.

In general, this time need not be the same as the moment at which the payment actually takes place.

14.21 There are two valuation principles in the national accounts, one for suppliers and one for users.
For suppliers, transactions in goods and services are to be valued at basic prices.

The basic price is the price per unit of good or service receivable by the producer.

We use the term receivable to indicate that the price refers to an accrued transaction for the seller, and the term pay- able to indicate a transaction that has accrued to the purchaser.
As the producer does not receive taxes (if any) on products, but does receive subsidies (if any) on pro- ducts, taxes on products are excluded from the basic price, while subsidies on products are included.

The producer also does not receive separately invoiced transport and insurance charges provided by other sup- pliers, or any distribution margins added by other, retail or wholesale service producers, and these are also ex- cluded from the basic price.
In contrast, the user, as purchaser, pays all of these charges, and users’ purchases are therefore valued at purchasers’ prices, which add taxes net of subsidies on products and margins for included transport, insurance and distribution services to the basic price.

14.22 The SNA 1993 distinguishes between taxes on products and other taxes on production.

Taxes net of subsidies on products T include all taxes payable per unit or as a fraction of the value of goods or services trans- acted.
Included in T are excise, sales, and the non- refundable portion of value added taxes, duties on imports and taxes on exports.

Subsidies on products include all subsidies receivable per unit or as a fraction of the value of goods or services produced, including in particular subsidies paid on imports and exports.

Other taxes on production comprise, for example, taxes on real property and taxes on proﬁts.
Other subsidies on pro- duction include, for example, regular payments by the government to cover the diﬀerence between the costs and revenues of loss-making enterprises.

Of total taxes and subsidies on production, only taxes and subsidies on products are considered in deﬁning basic and purchasers’ prices.
By implication, there are no taxes payable on products included in either of the aggregates Y or M, THE SYSTEM OF PRICE STATISTICS while subsidies receivable on products are included in these aggregates.

14.23 Accordingly, output Y and imports M in equations (14.1) and (14.2) are valued at basic prices, to which are added taxes less subsidies on products T to arrive at total supply.
The reader may have noted that transport, insurance and distribution margins have some- how disappeared after having been introduced.

Whether these services are included with the good or invoiced separately does not aﬀect the total expenditure on goods and services by the purchaser.

For the economy as a whole, these transactions cancel out, but when we con- sider industry or activity and product detail, they will have redistributive eﬀects among goods and services products.
This point is revisited in the discussion of the supply and use table below.

14.24 The components of total uses are valued at purchasers’ prices.

This is straightforwardly interpreted for the ﬁnal consumption of households and govern- ment.

For capital formation expenditures, the notion of purchasers’ prices also includes the costs of ‘‘setting up’’ ﬁxed capital equipment.
For exports, purchasers’ prices also include export taxes net of subsidies, according to the ‘‘free on board’’ (fob) value at the national frontier.

We now discuss each of the four major goods and ser- vices accounts in turn.

Production 14.25 An institutional unit engaged in production is said to be an enterprise.

By implication, any of the ﬁve types of resident institutional units can be an enterprise.
The production account for enterprises in the SNA 1993 appears, with minor reordering of elements, essentially as shown in Table 14.1.

An identical presentation also applies to the establishments or local kind of activity units (LKAUs) owned by enterprises.

In fact, an estab- lishment can be deﬁned operationally as the smallest unit for which a production account can be constructed.
There are cases in which an establishment or LKAU is synonymous with or at least inseparable from the institutional unit that owns it.

This is true of single- establishment corporations and of household unin- corporated enterprises, for example.

In other cases, an enterprise may own multiple establishments.
The Table 14.1 Production account for an establishment, institutional unit or institutional sector Uses Resources P.2 Intermediate consumption (purchasers’ prices) P.1 Output (basic prices) SNA 1993 items in bold refer to ﬂows in goods and services B.1 Gross value-added (balances the account; that is, it is the diﬀerence between output P.1 and intermediate consumption P.2) Of which, memorandum items breaking down total output for classifying the market/non-market status of the producer unit: P.11 Market output P.12 Output for own ﬁnal use P.13 Other non-market output 239 CONSUMER PRICE INDEX MANUAL: THEORY AND PRACTICE production account can also be produced for various including, of establishment and enterprise groupings, course, institutional sectors, but also for establishment industry or activity groups.
In the production account and throughout the SNA 1993, the transaction codes beginning with P refer to entries for transactions in goods and services.

The codes beginning with B refer to so- called ‘‘balancing items’’, which are deﬁned residually as the diﬀerence between a resources total and the sum of itemized uses of those resources.
14.26 For classifying an establishment or LKAU, output is broken down into market output (P.11), which is sold at ‘‘economically signiﬁcant prices’’ substantially covering the cost of production, and two types of non- market output that are provided without charge or at prices so low they bear no relationship to production cost.

The two types of non-market output are output for own ﬁnal use (P.12) and other non-market output (P.13).
Output for own ﬁnal use includes the production of, for example, machine tools and structures (ﬁxed capital formation items) by an establishment for the use of the establishment itself or other establishments in the same enterprise, the imputed rental value of certain productive assets owned by households, such as (and currently limited to) owner-occupied dwellings, and the produc- tion of certain other unincorporated household enter- prises, such as agricultural products produced by farmers for consumption by their own families or employees.
Other non-market output comprises the output of gen- eral government and non-proﬁt institutions serving households distributed free of charge or sold at prices that are not economically signiﬁcant.

In constructing a price index, we will necessarily be focusing on those transactions of establishment units that involve eco- nomically signiﬁcant prices, and thus on market output (P.11).
The prices collected for market output items may also, however, be used to value the own ﬁnal use portion of non-market output (P.12).

Our scope of coverage for price indices thus extends to cover this component of non-market output as well.

14.27 A production unit’s resources derive from the value of its output, and its uses of resources are the costs it incurs in carrying out production.
The production account therefore uses both the basic price and pur- chasers’ price methods of valuation, as appropriate to a production unit in its roles as a supplier and a user of products.

For the supply (resources) of goods and ser- vices, products are valued at basic prices, the national currency value receivable by the producer for each unit of a product.
The prices include subsidies, and exclude the taxes on products and additional charges or margins on products to pay for included retail and wholesale trade services, and for included transport and insurance.

For uses of goods and services, products are valued at purchasers’ prices, the national currency value payable by the user for each unit of a product, including taxes on products as well as trade and transport margins, and excluding subsidies on products.
14.28 Product detail in the production account.

In addition to breaking output down into its market and non-market components, output and intermediate con- sumption also can be broken down by type of product.

240 Classifying product types using, for example, the inter- national standard Central Product Classiﬁcation (CPC), the production account for each establishment could be arranged to appear as in Table 14.2.
Table 14.2 eﬀec- tively gives the core structure of the report form of the typical establishment survey providing source data on production for the national accounts.

14.29 Industry detail in the production account.

With the values of total output by product, and total market and non-market outputs in Table 14.2 for each estab- lishment, we then classify the establishment by its prin- cipal activity or industry, and market/non-market status.
To reﬂect the information required for this classiﬁcation, positions for the activity and market/non-market clas- siﬁcation codes of the establishment are shown at the top of Table 14.2.

The activity classiﬁcation involves principally, if not exclusively, sorting establishments according to the types of product produced (CPC or other product code, such as the Classiﬁcation of Pro- ducts by Activity) for which the total output is greatest.
The major categories of the International Standard Industrial Classiﬁcation of All Economic Activities (ISIC), Revision 3, are shown in Box 14.2 below.

14.30 As indicated in Table 14.2, The SNA 1993 recommends use of the International Standard Industrial Classiﬁcation (ISIC) for all economic activities, the CPC for domestic products, and the closely related Harmo- nized Commodity Description and Classiﬁcation System (HS) for exported and imported products.
Each country may adapt the international standard to its speciﬁc cir- cumstances.

If the adaptation amounts to adding further detail, the classiﬁcation is said to be derived from the international standard.

The Nomenclature ge´ne´rale des Activite´s e´conomiques dans les Communaute´s europe´ennes (NACE, the General Industrial Classiﬁcation of Eco- nomic Activities within the European Communities) is an industrial classiﬁcation derived from the ISIC.
If the adaptation reorganizes the way in which detailed categories are grouped compared with the interna- tional standard, but provides for a cross-classiﬁcation at some level of detail, it is said to be related.

The North American Industrial Classiﬁcation System (NAICS) of Canada, Mexico and the United States is an industrial classiﬁcation related to the ISIC.
The European Com- mission’s PRODCOM classiﬁcation of industrial pro- ducts is derived from its Classiﬁcation of Products by Activity (CPA) which, in turn, is related to the inter- national standard CPC through a cross-classiﬁcation deﬁned at a high level of product detail.

14.31 The output aggregate of the producer price index and the production account.

The producer price index (PPI) is an index of the prices of the outputs of estab- lishments.
The position of the PPI in the SNA 1993 is deﬁned by the relationship of its output value aggregate to those deﬁned in the national accounts.

In Box 14.2, we consider the composition of the PPI value aggregate according to its industry coverage, arguing that the PPI’s industry coverage should be complete.
Considering fur- ther market and non-market production within an indus- try group of establishments that are classiﬁed according to market status, the PPI’s coverage could extend both to the market output (P.11) and output for own ﬁnal use THE SYSTEM OF PRICE STATISTICS Table 14.2 Production account with product detail for an establishment or local kind of activity unit Establishment ID: eeeeeeee Activity/Industry code (ISIC): aaaa Uses SNA 1993 items in bold refer to ﬂows in goods and services Institutional unit ID: uuuuuuuu Institutional sector code: S.nnnnn Market status: P.1n Resources P.2 Intermediate consumption (purchasers’ prices), P.1 Output (basic prices), of which: of which: CPC 0 Agriculture, forestry and ﬁshery products CPC 1 Ores and mineral; electricity, gas, and water CPC 2 Food products, beverages and tobacco; textiles, apparel and leather products CPC 3 Other transportable goods, except metal products, machinery and equipment CPC 4 Metal products, machinery and equipment CPC 5 Intangible assets; land; constructions; construction services CPC 6 Distributive trade services; lodging; food and beverage serving services; transport services; and utilities distribution services CPC 7 Financial and related services; real estate services; and rental and leasing services CPC 8 Business and production services CPC 9 Community, social and personal services B.1 Gross value-added CPC 0 Agriculture, forestry and ﬁshery products CPC 1 Ores and mineral; electricity, gas, and water CPC 2 Food products,
 beverages and tobacco; textiles, apparel and leather products CPC 3 Other transportable goods, except metal products, machinery and equipment CPC 4 Metal products, machinery and equipment CPC 5 Intangible assets; land; constructions; construction services CPC 6 Distributive trade services; lodging; food and beverage serving services; transport services; and utilities distribution services CPC 7 Financial and related services; real estate services; and rental and leasing services CPC 8 Business and production services CPC 9 Community, social and personal services Memorandum items breaking down total output for classifying the market/non-market status of the producer: P.11 Market output P.12 Output for own ﬁnal use P.13 Other non-market output (P.12) identiﬁed in Table 14.2 when this account is con- sidered for all establishments in the economy.
Although the latter is technically non-market output, it would be valued at the basic prices the establishment would receive were that own-use production to be sold.

14.32 The expenditure aggregate of the consumer price index and the production account.

Consumption from own production is a signiﬁcant fraction of total consumption, comprising both goods and services.
For goods produced by households, as noted in the SNA 1993: 6.24 The System includes the production of all goods within the production boundary.

At the time the pro- duction takes place it may not even be known whether, or in what proportions, the goods produced are destined for the market or for own use.
The following types of pro- duction by households are, therefore, included whether intended for own ﬁnal consumption or not: (a) The production of agricultural products and their subsequent storage; the gathering of berries or other uncultivated crops; forestry; wood-cutting and the col- lection of ﬁrewood; hunting and ﬁshing; (b) The production of other primary products such as mining salt, cutting peat, the supply of water, etc.
; (c) The processing of agricultural products; the pro- duction of grain by threshing; the production of ﬂour by milling; the curing of skins and the production of leather; the production and preservation of meat and ﬁsh prod- ucts; the preservation of fruit by drying, bottling, etc.

; the production of dairy products such as butter or cheese; the production of beer, wine, or spirits; the production of baskets or mats; etc.
; (d) Other kinds of processing such as weaving cloth; dress making and tailoring; the production of footwear; the production of pottery, utensils or durables; making furniture or furnishings; etc.

The storage of agricultural goods produced by households is included within the production boundary as an extension of the goods-producing process.

The supply of water is also considered a goods-producing activity in this context.
In principle, supplying water is a similar kind of activity to extracting and piping crude oil.

6.25 It is not feasible to draw up a complete, exhaustive list of all possible productive activities but the above list covers the most common types.

When the amount of a good produced within households is believed to be quan- titatively important in relation to the total supply of that good in a country, its production should be recorded.

Otherwise, it is not worthwhile trying to estimate it in practice.
For services, the SNA 1993 notes housing services as the sole – but for most countries extremely important – item of production for own consumption: 6.29 The production of housing services for their own ﬁnal consumption by owner-occupiers has always been included within the production boundary in national accounts, although it constitutes an exception to the general exclusion of own-account service production.
The ratio of owner-occupied to rented dwellings can vary sig- niﬁcantly between countries and even over short periods of time within a single country, so that both interna- tional and intertemporal comparisons of the produc- tion and consumption of housing services could be distorted if no imputation were made for the value of 241 CONSUMER PRICE INDEX MANUAL: THEORY AND PRACTICE Box 14.2 Coverage of industries or activities by the producer price index in terms of aggregate output value The principal economic activities of the International Standard Industrial Classiﬁcation of All Economic Activities (ISIC), Revision 3, are: A Agriculture, hunting and forestry B Fishing C Mining and quarrying D Manufacturing E Electricity, gas and water supply F Construction G Wholesale and retail trade; repair of motor vehicles, motorcycles, and personal and household goods H Hotels and restaurants I J K Real estate, renting and business activities L M Education N Health and social work O Other community, social and personal service activities P Private households with employed persons Q Extra-territorial organizations and bodies Transport, storage and communications Financial intermediation Public administration and defence; compulsory social security These are characteristic of the activities identiﬁed in most national industrial classiﬁcations.
In assembling data on the supply and use ﬂows in the economy, a detailed industry production account such as given in Table 14.2 is effectively constructed for each type of activity in the economy, whose major categories are shown in the ISIC list above.

With the product output and expenditure detail in Table 14.2, we can show more explicitly the typical goods and services coverage of the PPI within the output aggregate (P.1) of the production account for each industry.
In most countries, PPIs cover industries that produce goods, such as the mining and manufacturing activities (C–D) and some- times also agriculture (A) and ﬁshing (B), and construction (F), as well as the two industrial service activities – electricity, gas and water supply (E) and transport, storage and communications (I).
In principle, the PPI should cover the market output of all activities, and a number of countries are currently working on extending PPI coverage to the remaining service-producing activities besides transport and utilities.

own-account housing services.

The imputed value of the income generated by such production is taxed in some countries.

The SNA imputes the value of such consumption at the equivalent market value of the output households pro- duce for their own purposes.
14.33 In some cases, however, the market equivalent method of valuing production for own consumption is not viable because suﬃciently similar market equiva- lents to the items supplied from own production are not available, or they are suﬃciently rare that it is too expensive to obtain information on them or too unreli- able to base estimates on such information.

In these cases, production cost approaches are taken.
The source of data for the production cost approaches is, in part, the household production account as regards goods and services purchased for intermediate consumption.

The ultimate source of primary information for the household production account is principally the house- hold expenditure survey, though specialized surveys of household business activity may also be undertaken for this purpose.
For the shelter services provided by houses occupied by their owners, for example, the production account would be the source of expenditures on utilities, maintenance, and do-it-yourself repair items of inter- mediate consumption that would be used, in part, to determine the cost to an owner-occupant of the services he or she derives from his or her own dwelling.
For the own production of agricultural produce, purchases of seed, fertilizer, and small garden tools might be recorded as intermediate consumption.

Particularly for the latter, however, it is often diﬃcult to distinguish between the intermediate production expense of production for own consumption and ﬁnal consumption expenditure for maintaining decorative landscaping.
Final consumption 14.34 Consumption of goods and services in the SNA 1993 is shown in the use of income account, which appears essentially as in Table 14.3 for each institutional unit.

It is recalled that the accounts pertaining to goods and services in the SNA 1993 that can be decomposed into price and volume components, and that would thus draw our interest as price index compilers, are desig- nated by the codes P.n.
Items of ﬁnal consumption are designated by P.3 with extensions: P.3 comprises indi- vidual consumption expenditure (P.31) and collective consumption expenditure (P.32).

14.35 Individual consumption, actual consumption, and household consumption expenditures.

The SNA dis- tinguishes individual from collective goods and services, a distinction that is equivalent to that between private and public goods in economic theory.

The distinction is mainly relevant to services.


Individual services are provided to individual households and beneﬁt those particular households, whereas collective services are provided to the community, for example services such as public order, administration, security and defence.

Many individual services, however, such as education, health, housing and transport, may be ﬁnanced and paid for by government or non-proﬁt institutions and provided free or at a nominal price to individual house- holds.
A large part of government consumption expen- diture is not on public goods but on goods or services supplied to individual households.

These individual consumption expenditures by governments and NPISHs are described as social transfers in kind in the SNA 1993.

14.36 The concept of ‘‘household consumption’’ can have three distinct meanings.
First, it can mean the total set of individual consumption goods and services actu- ally acquired by households, including those received as 242 Table 14.3 Use of income account for institutional units and sectors Institutional unit ID: uuuuuuuu Institutional sector code: S.nnnnn SNA 1993 items in bold refer to ﬂows in goods and services Uses Resources P.3 Final consumption expenditure (purchasers’ prices)1 B.6 Disposable income2 THE SYSTEM OF PRICE STATISTICS P.31 Individual consumption expenditure, of which: P.311 Individual consumption expenditure, except from production on own account, and imputed consumption expenditure, household sector S.14 only P.312 Imputed expenditure on owner-occupied housing services, household sector S.14 only P.313 Financial intermediation services implicitly measured (FISIM) P.314 Other imputed individual consumption expenditure P.32 Collective consumption expenditure (general government sector S.13 only) D.8 Adjustment for the change in the net equity of households in pension funds3 B.8 Saving (balances the account; that is, it is the diﬀerence between disposable income B.6 and the sum of expenditures P.3 and adjustment D.8) 1By deﬁnition, corporations have no ﬁnal consumption in the SNA 1993.
Thus, item P.3 and its subdivisions appear with non-zero entries only for household, government, and non-proﬁt institutions serving households (NPISH) units.

2 The SNA 1993 derives disposable income in a sequence of accounts producing the balancing items: value added B.1 (production account), operating surplus B.2 and mixed income B.3 (generation of income account), balance of primary incomes B.5 (allocation of primary income account), and disposable income B.6 (secondary distribution of income account).
Collapsing all of these steps, disposable income B.6 is value added B.1 less (net) taxes on production and imports (payable) D.2 plus (net) subsidies D.3 (receivable), plus compensation of employees receivable, plus (net) property income (receivable) D.4, less (net) taxes on income and wealth (payable) D.5, less (net) social contributions (payable) D.61, plus (net) social beneﬁts (receivable) D.62, less (net) other transfers (payable) D.7.
3 This adjustment reﬂects the treatment by the SNA 1993 of privately funded pensions as owned by the household beneﬁciaries of such plans.

It maintains consistency between the income and accumulation accounts in the system.

It is not relevant to price and volume measurement (see System of National Accounts 1993, Chapter IX, Section A.4 for further details).

social transfers in kind.

Second, it can mean the subset which households actually pay for themselves.
To dis- tinguish between these two sets, the SNA describes the ﬁrst as the actual ﬁnal consumption of households and the second as household ﬁnal consumption expenditures.

A third possible interpretation of household consump- tion is that it means the actual physical process of consuming the goods and services.

It is this process from which utility is derived and that determines the house- hold’s standard of living.
The process of consuming or using the goods or services can take place some time after the goods or services are acquired, as most con- sumer goods can be stored.

The distinction between acquisition and use is most pronounced in the case of consumer durables that may be used over very long periods of time.

The treatment of durables is discussed further in Box 14.3.
14.37 The existence of social transfers in kind is not generally recognized in CPIs, although it is desirable to take account of them, especially when considering changes in the cost of living.

Moreover, governments may start to charge for services that were previously provided free, a practice that has become increasingly common in many countries in recent years.
The goods and services provided free as social transfers could, in principle, be regarded as also being part of house- hold consumption expenditures but as having a zero price.

The shift from a zero to positive price is then a price increase that could be captured by a consumer price index.

14.38 Monetary and imputed expenditures.

Not all household expenditures are monetary.


A monetary expenditure is one in which the counterpart to the good or service acquired is the creation of some kind of ﬁnancial liability.

This may be immediately extinguished by a cash payment, but many monetary expenditures are made on credit.

Household consumption expenditures also include certain imputed expenditures on goods or services that households produce for themselves.
These are treated as expenditures because households incur the costs of producing them (in contrast to social transfers in kind, which are paid for by government or non-proﬁt institutions).

14.39 The imputed household expenditures recog- nized in the SNA include all those on goods that house- holds produce for themselves (mainly agricultural goods in practice), but exclude all household services produced for own consumption except for housing services pro- duced by owner-occupants.
The imputed prices at which the included goods and services are valued are their estimated prices on the market.

In the case of housing services, these are imputed market rentals.

In practice, most countries follow the SNA by including owner- occupied housing in the CPI.

Other imputed prices, such as the prices of vegetables, fruit, or dairy or meat pro- ducts produced for own consumption, may be included if they comprise a suﬃciently large component of household consumption expenditure.
14.40 Product detail in the use of income account.

As with the production accounts of the establishments owned by institutional units, we can consider extending the product detail of goods and services consumption in the use of income account according to the type of pro- duct consumed.
In order to maintain the integration of the system of price and volume statistics on consumption with those we have just covered on production, products 243 CONSUMER PRICE INDEX MANUAL: THEORY AND PRACTICE Box 14.3 Treatment of housing and consumer durables in the system of national accounts and in consumer price indices Dwellings are ﬁxed assets.

Purchases of dwellings by households therefore constitute household gross ﬁxed capital formation and are not part of household consumption.
They cannot enter into a price index for household consumption.

Fixed assets are used for purposes of production, not consumption.

Dwellings have therefore to be treated as ﬁxed assets that are used by their owners to produce housing services.

The system of national accounts (SNA) actually sets up a production account in which this production is recorded.

The services are consumed by the owners.


The expenditures on the services are imputed, the services being valued by the estimated rentals payable on the market for equivalent accommodation.

The rentals have to cover both the depreciation of the dwellings and the associated interest charges or capital costs.
The existence of these imputed expenditures on owner- occupied housing services has always been recognized in national accounts and most countries have also included them in their consumer price indices (CPIs), even though other imputed expenditures are not included.

Consumer durables, such as automobiles, cookers and free- zers, are also assets that are used by their owners over long periods of time.
In principle, they could be treated in the same way as dwellings and be reclassiﬁed as ﬁxed assets that produce ﬂows of services that are consumed by their owners.

For certain analytic purposes, it may be desirable to treat them this way.

To do so in the SNA, however, would not simply be a matter of estimating the market rentals that would be payable for hiring the assets.

It would also be necessary to set up production accounts in which the durables are used as ﬁxed assets.
This has tradi- tionally been regarded as too difﬁcult and artiﬁcial.

There are also objections to extending further the range of imputed ﬂows inclu- ded in the SNA and gross domestic product.

In practice, there- fore, expenditures on durables are classiﬁed in the SNA as consumption expenditures and not as gross ﬁxed capital forma- tion, a practice that is carried over into CPIs.

would be classiﬁed according to the same system as in the production account.
We show the major categories of the CPC 1.0 within the components of ﬁnal consumption expenditure in Table 14.4.

14.41 Although the discussion in this chapter main- tains a consistent classiﬁcation of expenditure by product across all goods and services accounts, other functional classiﬁcations of expenditure have been developed for each institutional sector for speciﬁc purposes.
The international standard versions of these classiﬁcations included in the SNA 1993 comprise the Classiﬁcation of Individual Consumption according to Purpose (COI- COP), the Classiﬁcation of the Purposes of Non-proﬁt Institutions Serving Households (COPNI), the Classiﬁ- cation of the Functions of Government (COFOG), and the Classiﬁcation of the Purposes of Producers (COPP).

The ﬁrst column of Tables 14.4 and 14.5 is often com- piled using data from household expenditure surveys.
These data are collected using functional classiﬁcations such as COICOP, rather than product classiﬁcations.

To facilitate constructing the cross-economy framework of the SNA 1993 considered in this chapter, there is a concordance between the CPC and the COICOP.

14.42 A hierarchy of household consumption aggre- gates.

It is worth noting that all household consumption 244 expenditures (that is, of the households institutional sector S.14) are individual expenditures, by deﬁnition.
The following hierarchy of household consumption aggre- gates that are relevant to CPIs may be distinguished in the SNA: P.41 Actual individual consumption, of which: D.63 Social transfers in kind (the individual con- sumption expenditure P.31 of general gov- ernment S.13 and NPISHs S.15) P.31 Individual consumption expenditure, of which: P.311 Monetary consumption expenditure P.312 Financial intermediation services im- plicitly measured (FISIM) P.313 Imputed expenditure on owner- occupied housing services P.314 Other imputed individual consumption expenditure The codes P.311, P.312, P.313 and P.314 do not exist in the SNA 1993 but are introduced for convenience here.
These four sub-categories of household consumption expenditures are separately identiﬁed in Tables 14.4 and 14.5.

As already noted, D.63 is usually excluded from the expenditure coverage of CPIs.

14.43 It is worth noting the special treatment of ﬁnancial services in the SNA 1993.

FISIM comprises expenditures on those market services provided by ﬁnan- cial institutions that are not separately distinguished from interest charges.
Expenditures on ﬁnancial services on which there is an explicit charge are already covered in P.311.

Although FISIM P.312 requires an implicit measurement as the diﬀerence between a market interest rate and a reference rate, it is part of an observed interest payment and thus is not considered an imputed expen- diture in the same sense as imputed rent P.313 and other imputed expenditure P.314.
14.44 Our item P.314, other imputed individual consumption, includes, besides households’ production of goods for their own consumption, expenditures on goods and services that employers make on behalf of their employees as non-cash compensation.

The SNA calls this item D.12, employers’ social contributions, and considers it in the generation of income account.
It is recognized as a component of the labour services price index, but is not customarily included in the CPI despite its dual role as an item of consumption (see paragraph 14.75 below).

14.45 The expenditure aggregate of the consumer price index and the use of income account.
The detailed use of income accounts for institutional sectors can be assembled into a consolidated framework by choosing columns from Table 14.4 for each sector and displaying them together as in Table 14.5, which gives an economy- wide presentation of ﬁnal consumption and saving.

Table 14.5, for the total economy, shows individual consumption as comprising the individual consumption entries P.31 of the use of income accounts for house- holds, NPISHs and the general government sector.
It also aggregates the disposable income B.6 of all three.

It shows separately the ﬁnal collective consumption of government P.32.

The account in Table 14.5 has been arranged speciﬁcally to show the consumption coverage l e b a s o p s D i 6 .

B n o i t p m u s n o c l a n F i 3 .

P n o i t p m u s n o c e v i t c e l l o C 2 3 .

 .
, ; ) s e c r u o s e R ( n m u o c l t h g i r r a f f o l i a t e d w o h s ) s e s U ( .
4 1 l e b a T s e c r u o s e R n n n n n .

S : : D I t i n u l a n o i t u t i t s n I s e s U e m o c n i ) s e c i r p ’ s r e s a h c r u p , l a t o t ( .

P l i a u d v d n i i r e h t O 4 1 3 .

P e r u t i d n e p x e d e t u p m I 2 1 3 .

P n o i t p m u s n o c y r a t e n o M 1 1 3 .

P l y n o 3 1 .
P l y n o 3 1 .

S t n e m n r e v o g l a r e n e g : - .
P e r u t i d n e p x e n w o ’ , - n i n o i t p m u s n o c , .

D i g n v a S 8 .
D i g n v a S 8 .

 , y t i n u m m o C 9 C P C d n a l i a c o s , , y t i n u m m o C 9 C P C ’ s r e y o p m E l ‘ ‘ 2 1 .
D s e s i r p m o c ’ ’ ‘ ‘ , y t i n u m m o C 9 C P C i s e c v r e s l i a c d e m , e r a c d l i h c , t r o p s n a r t i , g n s u o h f o i i n o s v o r p e r a ’ ’ s n o i t u b i r t n o c l i a c o s ’ s r e y o p m E l ‘ ‘ g n o m A .
.

D .

’ ’ .
, ’ ’ s n o i t u b i r t n o c l i a c o s ’ s r e y o p m E l ‘ ‘ i .

s e c v r e s e c n a r u s n i e f i l d n a i , s e c v r e s d n a e c n a r u s n i .
, , e t a t s e l a e r e h t o t n o i t i d d a n I 2 i .
, e r u t l u c i r g A 0 C P C d n a y r t s e r o f , e r u t l u c i r g A 0 C P C y r t s e r o f , ; l a r e n m d n a s e r O 1 C P C i ; l a r e n m d n a s e r O 1 C P C , y t i c i r t c e e l i ; l a r e n m d n a s e r O 1 C P C r e t a w d n a , s a g , y t i c i r t c e e l r e t a w d n a , s a g , y t i c i r t c e e l r e t a w d n a , s a g s e g a r e v e b , s t c u d o r p d o o F 2 C P C s e g a r e v e b , s t c u d o r p d o o F 2 C P C s e g a r e v e b , s t c u d o r p d o o F 2 C P C l e r a p p a , s e l i t x e t ; o c c a b o t d n a , s e l i t x e t ; o c c a b o t d n a , s e l i t x e t ; , , , s t c u d o r p l a t e m t p e c x e , s d o o g l e b a t r o p s n a r t r e h t O 3 C P C , s d o o g l e b a t r o p s n a r t r e h t O 3 C P C , s d o o g l e b a t r o p s n a r t r e h t O 3 C P C , y t i c i r t c e e l i ; ﬁ
, e r u t l u c i r g A 0 C P C r e t a w d n a , s a g d n a l e r a p p a , s e l i t x e t ; o c c a b o t d n a s e g a r e v e b , s t c u d o r p d o o F 2 C P C , , s t c u d o r p l a t e m t p e c x e i y r e n h c a m , s t c u d o r p l a t e M 4 C P C i y r e n h c a m , s t c u d o r p l a t e M 4 C P C i y r e n h c a m , s t c u d o r p l a t e M
 4 C P C d n a i y r e n h c a m , ; g n g d o l t r o p s n a r t i ; s e c v r e s i g n v r e s s e i t i l i t u d n a i ; ; g n g d o l t r o p s n a r t i ; s e c v r e s i g n v r e s s e i t i l i t u d n a i ; ; g n g d o l t r o p s n a r t i ; s e c v r e s i g n v r e s s e i t i l i t u d n a i ; s e c v r e s i s e c v r e s n o i t u b i r t s d i i ; s e c v r e s e d a r t e v i t u b i r t s D i 6 C P C i ; s e c v r e s e d a r t e v i t u b i r t s D i 6 C P C i ; ; g n g d o l t r o p s n a r t i ; s e c v r e s i g n v r e s s e i t i l i t u d n a i ; s e c v r e s i s e c v r e s n o i t u b i r t s d i i ; ) I M S F ( I d e r u s a e m y l t i c i l p m i i ; ; ; ; s
; ; s e c v r e s e t a t s e l a e r l a t n e r d n a i ; ; s e c v r e s l a t n e r d n a i ; ; ( d n
c n a n ﬁ a h t i w s t i s o p e d p e e k ( , e k i l i e h t d n a s e c v r e s y r o s v d a , n a o i l , .
- - t e k r a m e m a s e h t f o e r a , , s e r u t i d n e p x e d e t u p m i e s e h t l i , e p c n i r p n I .

e t a r e c n e r e f e r a t a w o r r o b r o ) n o i t u t i t s n i 245 y m o n o c e l a t o T , e m o c n i l e b a s o p s D 6 .
B i h c i h w f o , 1 .

S y m o n o c e ’ s r e s a h c r u p ( 1 .

S y m o n o c e l a t o t , e r u t i d n e p x e l a t o T , e r u t i d n e p x e n o i t p m u s n o c l a n F i 3 .

P n o i t p m u s n o c e v i t c e l l o C 2 3 .

P : g n i s i r p m o c , ) s e c i r p 3 1 .

S r o t c e s t n e m n r e v o g s r o t c e s 5 1 .

S I 1 H S P N d n a 3 1 .


S l a r e n e g , e r u t i d n e p x e t n e m n r e v o g l a r e n e g , .

P n o i t p m u s n o c l a u d i v i d n I 1 3 .

P d n k i n i s r e f s n a r t l i a c o S 3 6 .

D l i a u d v d n i i d e t u p m i r e h t O 4 1 3 .

P n o e r u t i d n e p x e d e t u p m I 3 1 3 .

P n o i t p m u s n o c y r a t e n o M 1 1 3 .

P 4 1 .
P 4 1 .

S r o t c e s l d o h e s u o H , .

P 2 1 # : g n s i r p m o c , ) s e c i r p ’ s r e s a h c r u p ( 1 .

S y m o n o c e l a t o T , .

P i : g n s i r p m o c s e s u h t i w , 1 .


 ; .

S : : D I t i n u l a n o i t u t i t s n I : - r e n w o l 4 1 .
P r e t a w d n a , s a g , y t i c i r t c e e l , s a g , y t i c i r t c e e l r e t a w d n a r e t a w d n a , s a g r e t a w d n a , s a g s t c u d o r p r e h t a e l , l e r a p p a s t c u d o r p r e h t a e l , , s e l i t x e t ; o c c a b o t , s e l i t x e t ; o c c a b o t l e r a p p a , s e l i t x e t ; o c c a b o t d n a l e r a p p a , s e l i t x e t ; o c c a b o t d n a , s e g a r e v e b , s t c u d o r p d o o F 2 C P C , s e g a r e v e b , s t c u d o r p d o o F 2 C P C s e g a r e v e b , s t c u d o r p d o o F 2 C P C s e g a r e v e b , s t c u d o r p d o o F 2 C P C i ; l a r e n m d n a s e r O 1 C P C i ; l a r e n m d n a s e r O 1 C P C , y t i c i r t c e e l i ; l a r e n m d n a s e r O 1 C P C , y t i c i r t c e e l i ; l a r e n m d n a s e r O 1 C P C , , s d o o g l e b a t r o p s n a r t r e h t O 3 C P C , , , s d o o g i y r e n h c a m , , , s t c u d o r p l a t e m t p e c x e i y r e n h c a m , s t c u d o r p l a t e M 4 C P C , s t c u d o r p l a t e M 4 C
 P C i y r e n h c a m , s t c u d o r p l a t e M 4 C P C i y r e n h c a m , , e r u t l u c i r g A 0 C P C d n a y r t s e r o f , e r u t l u c i r g A 0 C P C d n a y r t s e r o f , e r u t l u c i r g A 0 C P C d n a y r t
 s e r o f , e r u t l u c i r g A 0 C P C , y t i c i r t c e e l i ; l a r e n m d n a s e r O 1 C P C r e t a w d n a , s a g d n a y r t s e r o f , , s e l i t x e t ; , , s t c u d o r p l a t e m t p e c x e , , ; g n g d o l t r o p s n a r t i ; s e c v r e s i g n v r e s s e i t i l i t u d n a i ; ; g n g d o l i ; ; ; s e c v r e s e g a r e v e b d n a d o o f i ; g n g d o l t r o p s n a r t i ; s e c v r e s i g n v r e s s e i t i l i t u d n a i ; ; g n g d o l i ; s e c v r e s i ; s e c v r e s i g n v r e s e g a r e v e b d n a i ; ; g n g d o l t r o p s n a r t i ; s e c v r e s i g n v r e s s e i t i l i t u d n a i ; s e c v r e s i s e c v r e s n o i t u b i r
 t s d i i ; ; ; s e c v r e s e d a r t e v i t u b i r t s D i 6 C P C ) I M S F ( .
S n i e g n a h c e h t r o f t n e m t s u d A j 8 .

D 1 .

S y m o n o c e l a t o T i , g n v a S 8 .
; s e c v r e s l a t n e r d n a i ; ; s e c v r e s l a t n e r d n a i ; ; s e c v r e s e t a t s e l a e r i ; s e c v r e s e t a t s e l a e r i ; , y t i n u m m o C 9 C P C l i a c o s , y t i n u m m o C 9 C P C l i a c o s , y t i n u m m o C 9 C P C l i a c o s , y t i n u m m o C 9 C P C e t a t s e l a e r i ; s e c v r e s l a t n e r d n a i ; ; s e c v r e s e t a t s e l a e r i ; , ; d
4 1 l e b a T r o f e p o c s n i e r a M S F I I l .

 ) I I M S F ( .

P 2 l .
P 2 l .

 - .
- .

e p m a x e r o f , s I P C n o i t c a s n a r t r o n o i t a ﬂ n i .
n o i t a v r e s b o THE SYSTEM OF PRICE STATISTICS of the typical CPI, which comprises the ﬁrst and second columns and is labelled CPI reference aggregate #1.

This aggregate corresponds with the practice of most, but not all, countries and comprises, as shown in Table 14.5, the monetary (non-imputed) individual consumption expen- diture of the household sector (P.311) plus the implicit rent paid by homeowners on their own residences (P.313).
Box 14.3 contains further discussion on housing and durables in the CPI consumption expenditure aggregate.

Capital formation 14.46 Capital formation comprises: the accumula- tion of ﬁxed tangible and intangible assets, such as equipment, structures and software; changes in inven- tories and work in progress; and acquisitions less dis- posals of valuables, such as works of art.
These items are accounted for in the SNA capital account, which appears, with minor resorting, essentially as in Table 14.6 for each institutional unit.

Net lending (+)/net borrowing () is the balancing item of the capital account, making the uses on the left, comprising net acquisitions of stocks of various tangible and intangible items, add up to the resources on the right, comprising the sources of income ﬁnancing them.
From our earlier discussion on institutional units and establishments, it would be easy to conclude that the smallest eco- nomic unit to which the capital account can apply is the institutional unit.

It was asserted earlier that only institutional units maintain balance sheets and can mon- itor the stock variables that are the focus of this account.
Nevertheless, the physical capital assets for which changes are tracked in the capital account can and should be compiled, if possible, at the establishment/LKAU.

Such data are particularly useful for productivity analysis, even though complete capital accounts cannot be com- piled at the establishment level.

14.47 Product detail in the capital account.


As with the other goods and services-related accounts in the SNA 1993, the capital account’s goods and services items, designated by the codes P.5 with extensions, can be expanded by product type.

The account therefore can be rearranged to show details of goods and services as in Table 14.7, which, as Table 14.6, may pertain to an institutional unit, an institutional sector aggregate, or the total economy.
For an institutional unit, Table 14.6 contains the core set of items in the report form of the typical capital the national formation survey for accounts.

Our focus is on the CPI here, and thus on the version of the form that typically would be part of the package a respondent would ﬁll out in a house- hold expenditure survey.
In addition to the Central Product Classiﬁcation (CPC), version 1.0 shown here, the SNA 1993, Annex V contains a Non-ﬁnancial assets Table 14.6 Capital account Institutional unit ID: uuuuuuuu Uses P.5 Gross capital formation, of which: P.51 Gross ﬁxed capital formation Items in bold refer to ﬂows of goods and services P.511 Acquisitions less disposals of tangible ﬁxed assets P.5111 Acquisitions of new tangible ﬁxed assets P.5112 Acquisitions of existing tangible ﬁxed assets P.5113 Disposals of existing tangible ﬁxed assets P.512 Acquisitions less disposals of intangible ﬁxed assets P.5121 Acquisitions of new intangible ﬁxed assets P.5122 Acquisitions of existing intangible ﬁxed assets P.5123 Disposals of existing intangible ﬁxed assets P.513 Additions to the value of non-produced non-ﬁnancial assets P.5131 Major improvements to non-produced non-ﬁnancial assets P.5132 Costs of ownership transfer on non-produced non-ﬁnancial assets P.52 Change in inventories P.53 Acquisitions less disposals of valuables Institutional sector: S.nnnnn Resources B.10.1 Changes in net worth due to saving and capital transfers, of which: - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - K.1 Consumption of ﬁxed capital
 () - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - K.2 Acquisitions less disposals of non-produced non-ﬁnancial assets K.1 Consumption of ﬁxed capital () K.21 Acquisitions less
 disposals of land and other tangible non-produced assets K.22 Acquisitions less disposals of intangible non-produced assets B.8n Saving, net B.8 Saving (gross, from use of income account) B.9 Net lending ( +)/net borrowing () D.9 Capital transfers receivable (+) D.91 Investment grants D.92 Other capital transfers receivable D.9 Capital transfers payable () D.91 Capital taxes payable D.91 Other capital transfers payable 247 , s r e f s n a r t h t r o w t e n n i e g n a h C 1 .
0 1 .

 .

S : : .

 .

P s n o i t i s i u q c A 3 5 .

P 1 s e i r o t n e v n i n i e g n a h C 2 5 .


P n o i t a m r o f l a t i p a c d e x ﬁ s s o r G 1 5 .
P s e g a r e v e b , s t c u d o r p d o o F 2 C P C , s t c u d o r p d o o F 2 C P C , s t c u d o r p d o o F 2 C P C , s a g , y t i c i r t c e e l r e t a w d n a , s a g , y t i c i r t c e e l r e t a w d n a , s a g , , e r u t l u c i r g A 0 C P C y r t s e r o f , e r u t l u c i r g A 0 C P C y r t s e r o f , e r u t l u c i r g A 0 C P C i ; l a r e n m d n a s e r O 1 C P C i ; l a r e n m d n a s e r O 1 C P C i ; l a r e n m d n a s e r O 1 C P C , s e l i t x e t ; , s e l i t x e t ; , s e l i t x e t ; , s t c u d o r p l a t e m t p e c x e , s d o o g t n e m p u q e i l a t e m t p e c x e , s d o o g i y r e n h c a m , , s d o o g i y r e n h c a m , , s t c u d o r p l a t e M 4 C P C , s t c u d o r p l a t e M
 4 C P C , s t c u d o r p l a t e M 4 C P C - n o n d e c u d o r p - n o n f o e u l a v 5 : h c i h w f o , s t e s s a l a i c n a n ﬁ 4 : h c i h w f o , .
P s s e l s n o i t i s i u q c A 2 1 5 .

P s s e l s n o i t i s i u q c A 1 1 5 .
P 3 : h c i h w f o , , , s t c u d o r p l a t e M 4 C P C ; s t e s s a i l e b g n a t n I 5 C P C ; s t e s s a l i e b g n a t n I 5 C P C ; s t e s s a i l e b g n a t n I 5 C P C ; s t e s s a l i e b g n a t n I 5 C P C ; s t e s s a i l e b g n a t n I 5 C P C d e c u d o r p - n o n f o l s a s o p s d i s t e s s a l i a c n a n ﬁ - n o n / ) i ( g n w o r r o b t e N 9 .
B ) + ( .

K 2 .

K i s e c v r e s n o i t c u r t s n o c ; s n o i t c u r t s n o c ; d n a l i s e c v r e s n o i t c u r t s n o c ; s n o i t c u r t s n o c ; d n a l i s e c v r e s n o i t c u r t s n o c ; s n o i t c u r t s n o c ; d n a l i s e c v r e s n o i t c u r t s n o c ; s n o i t c u r t s n o c ; d n a l 4 1 .
S r o t c e s l d o h e s u o H , s e r u t c u r t s l a i t n e d s e R i a 1 1 5 .

 .

P i s e c v r e s n o i t c u r t s n o c ; s n o i t c u r t s n o c ; d n a l : h c h w i f O i l d e x ﬁ e b g n a T 1 1 1 .

N A e d o c t e s s a 3 9 9 1 A N S 3 i .

s e c v r e s n o i t c u r t s n o c d n a , s n o i t c u r t s n o c , d n a l , s t e s s a e b g n a t n i l i s e d u c x E l l l .

s e b a u a V 3 1 .
s e b a u a V 3 1 .

N A e d o c t e s s a 3 9 9 1 A N S 2 .

s n o i t c u r t s n o c d n a , d n a l , s t e s s a e b g n a t n l i i s e d u c x E l .

s e i r o t n e v n I 2 1 .

N A e d o c , s t e s s a e b g n a t n l i i s e d u c x E l .

s t e s s a d e c u d o r p - n o N 2 .

N A e d o c t e s s a 3 9 9 1 A N S 5 i .

s e c v r e s n o i t c u r t s n o c d n a , s n o i t c u r t s n o c , d n a l s e d u c x E l i l .

s t e s s a d e x ﬁ e b g n a t n I 2 1 1 .


N A e d o c t e s s a 3 9 9 1 A N S 4 i .

s e c v r e s n o i t c u r t s n o c d n a , d n a l , .

s t e s s a i s e c v r e s n o i t c u r t s n o c d n a , s n o i t c u r t s n o c classiﬁcation identifying the speciﬁc tangible, intangible, produced, and non-produced ﬁxed assets, as well as inven- tory and valuables items, recognized by the SNA 1993.
14.48 The expenditure aggregate of the CPI and the capital account.

The CPI may be deﬁned to include the household sector’s ﬁnal expenditure not only for con- sumption but also for capital formation.

This brings into the CPI expenditure aggregate the purchase of new residential structures or expenditure on major improve- ments to existing residential structures.
Consumer price index expenditure aggregate #2 is deﬁned as the monetary individual consumption expenditure of households P.311 in Table 14.5, which excludes all imputed expenditure, plus household expenditure on residential ﬁxed capital formation shown as item P.511a Residential structures, Household sector S.14 (shown in a box in Table 14.7).

External trade 14.49 The external account of goods and services is shown in Table 14.8.
It contains the transactions of the non-resident institutional units sector – S.2 Rest of the world – with the ﬁve types of resident units taken together and determines the trade deﬁcit (B.11) as imports (resources to rest of the world S.2) less exports (use or resources by the rest of the world).
The external goods and services account generally is taken from the balance of payments, which uses adjusted merchandise trade infor- mation from the customs services for goods P.61 and P.71, and assembles services data on P.62 and P.72 from various sources.

For further details, see Interna- tional Monetary Fund: Balance of payments manual (ﬁfth edition, 1993).
Although the account of external goods and services is shown as an aggregate of the external transactions of all resident institutional units by the SNA 1993, it may be possible to disaggregate it to distinguish the external goods and services expenditures of institu- tional sectors, hence the institutional sector designation S.1.nnnn at the top of Table 14.8 to include this possibility.
Our principal interest would be in the household sector S.14 and its subsectors S.14nn, as these would relate to the CPI.

14.50 Product detail in the external account of goods and services.

As with the other accounts, the external goods and services account can be expanded to show product detail, as in Table 14.9.

Regarding Table 14.9, the SNA 1993 states (SNA 1993, paragraph 15.68) that imported goods should be valued at cost-insurance- freight (cif ) at the level of detailed products.
On the other hand, the SNA 1993 requires that, in total, imports of goods be valued free-on-board (fob) at the border of the exporting country, thus excluding insurance and trans- port in a single adjustment to total imports of goods cif (SNA 1993 paragraphs 14.36–14.41).

That part of freight services on imports provided by non-residents is included in imports of transport services, and that part of insurance services provided on imports by non-resi- dents is added to imports of insurance services.
Trans- port and insurance services provided by residents on imports are included in exports of transport and insur- ance services.

This rather roundabout approach is taken to imports by product because, as a practical matter, it may be diﬃcult to obtain insurance and freight charges THE SYSTEM OF PRICE STATISTICS on imports from customs administrative data systems at the product level of detail (see SNA 1993, paragraphs 14.40–14.41).
Recent developments in computerized customs documentation have made the itemization of insurance and freight more straightforward, and the SNA 1993 does also allow for the possibility of deter- mining imports by product at their fob values, consistent with the aggregate valuation of imports.

If trade data are collected by a survey of resident institutional units, the core elements of the report form for such a survey would be as given in Table 14.9.
14.51 The export and import price indices and the external account of goods and services.

From the point of view of the residents of an economic territory, exports are a supply of goods and services to non-residents.

The SNA, however, records exports from the non-resident’s point of view, as a non-resident use of goods and services supplied by residents.

Accordingly, the relevant valua- tion principle for exports determining the behaviour of the non-resident user is the purchasers’ price.
The SNA takes the purchasers’ price to the non-resident user to be the fob price at the frontier of the resident supplier’s economic territory or country.

14.52 From the resident’s point of view, imports are a use of goods and services supplied by non-residents.

The SNA, however, records international trade from the non-resident’s point of view, as the supply of goods and services to residents by non-residents.
Accordingly, the relevant valuation principle for imports determining the behaviour of the non-resident supplier is the basic price.

The SNA takes the basic price to the non-resident sup- plier to be the fob price at the frontier of the non-resident supplier’s country in the rest of the world.
The supply and use table 14.53 Arraying elements of resources and uses from the production account, use of income account, capital account, and external accounts of goods and services in a particular conﬁguration, we can derive a format for the production portion of an analytical presentation of the data called a supply and use table (SUT).

An SUT is shown in Table 14.10.


It arrays various accounts relevant to monitoring developments in production and con- sumption within a country according to the supply and uses of goods and services.
14.54 In terms of the SNA 1993 codes, the supply of goods and services comes from: resident establishments (arranged in industries) in the form of domestic output (P.1), given by Y in equa- tions (14.1) and (14.2); the rest of the world as imports (P.7), given by M in equations (14.1) and (14.2), adjusted for trade and transport margins and taxes less subsidies on products (D.21–D.31), given by T in equations (14.1) and (14.2); and the uses of goods and services are for: current inputs into production by resident producers (arranged in industries) in the form of intermediate consumption (P.2), given by Z in equations (14.1) and (14.2); 249 CONSUMER PRICE INDEX MANUAL: THEORY AND PRACTICE Table 14.8 External account of goods and services Resident institutional units classiﬁed into sectors S.1.nnnn with non-resident institutional units S.2; SNA 1993 goods and services items shown in bold Uses Resources P.6 Exports of goods and services P.61 Exports of goods P.62 Exports of services B.11 External balance of goods and services P.7 Imports of goods and services P.71 Imports of goods P.72 Imports of services Table 14.9 External account of goods and services with product detail Resident institutional units classiﬁed into sectors S.1.nnnn with non-resident institutional units S.2; SNA 1993 goods and services items shown in bold Uses Resources P.6 Exports of goods and services Export price index uses aggregate P.61 Exports of goods At fob values CPC 0 Agriculture, forestry and ﬁshery products CPC
 1 Ores and mineral; electricity, gas, and water CPC 2 Food products, beverages and tobacco; textiles, apparel and leather products CPC 3 Other transportable goods, except metal products, machinery and equipment P.7 Imports of goods and services Import price index supply aggregate P.71 Imports of goods At fob values, of which: At cif values:1 CPC 0 Agriculture, forestry and ﬁshery products CPC 1 Ores and mineral; electricity, gas, and water CPC 2 Food products, beverages and tobacco; textiles, apparel and
 leather products CPC 3 Other transportable goods, except metal products, machinery and equipment CPC 4 Metal products, machinery and equipment CPC 4 Metal products, machinery and equipment P.62 Exports of services CPC 5 Intangible assets; land; constructions; construction services2 CPC 6 Distributive trade services; lodging; food and beverage serving services; transport services; and utilities distribution services, of which: Distributive trade services; lodging; food and beverage serving services; transport services; and utilities distribution services; except transport services on imports and exports rendered by residents Transport services on imports and exports rendered by residents Less: Adjustment to total imports of goods cif for insurance and freight provided by both residents and non-residents for delivery to the ﬁrst resident recipient.
P.72 Imports of services CPC 5 Intangible assets; land; constructions; construction services2 CPC 6 Distributive trade services; lodging; food and beverage serving services; transport services; and utilities distribution services, of which: Distributive trade services; lodging; food and beverage serving services; transport services; and utilities distribution services; except transport services on imports rendered by non-residents Transport services on imports and exports rendered by non-residents CPC 7 Financial and related services; real estate services; and rental and leasing services, of which: CPC 7 Financial and related services; real estate services; and rental and leasing services, of which: Financial and related services; real estate services; and rental and leasing services; except insurance services on imports rendered by residents Insurance services on imports rendered by residents Financial and related services; real estate services; and rental and leasing services; except insurance services on imports rendered by non-residents Insurance services on imports rendered by non-residents CPC 8 Business and production services CPC 9 Community, social and personal services CPC 8 Business and production services CPC 9 Community, social and personal services B.11 External balance of goods and services 1The SNA 1993 values imports fob, but it allows for the fact that while fob valuation by product would be consistent and preferred, compiling such data may be problematic at the product level of detail.
Imports of goods cif by product may be all that are available because the insurance and freight data are often not separately compiled by product in customs systems (see SNA 1993, paragraph 15.68).

Totals for these data may be obtained instead from resident and non- resident shippers in the process of compiling the balance of payments.

Insurance and freight services provided by residents on imports are a services export.
Regarding goods and services valuations in the import price and volume indices, see MPI in Tables 14.12 and 14.15, where it is explained that both fob and purchasers’ price valuations are important in constructing the MPI as a deﬂator for imports fob.

Imports at purchasers’ prices would be imports cif plus import tariﬀs, as well as domestic insurance and freight for delivery to the ﬁrst domestic owner.

2Construction services only.


250 i g n d a h s l a c i t r e v : ; g n d a h s l a n o g a d i : t n u o c c a l a t i p a C i ; g n d a h s o n d n a s e n i l t u o l e g n s i : t n u o c c a e m o c n i f o e s U i ; g n d a h s o n d n a s e n i l t u o l e b u o d : t n u o c c a n o i t c u d o r P ) T U S ( l e b a t e s u d n a l y p p u s e h T 0 1 .
4 1 l e b a T 251 CONSUMER PRICE INDEX MANUAL: THEORY AND PRACTICE ﬁnal domestic consumption, including individual consumption by resident households, resident non- proﬁt institutions serving households (NPISHs), and the government (P.31), and collective consumption by the government (P.32), given by, respectively, C and G in equations (14.1) and (14.2); capital formation by resident enterprises (P.5) (com- prising ﬁxed capital inventory change (P.52), and acquisitions less disposals of valu- ables (P.53)), given by I in equations (14.1) and (14.2); export (P.6) and use by the rest of the world, given by formation (P.51), X in equations (14.1) and (14.2).
14.55 Trade and transport margins do not appear in the standard sequence of accounts in the SNA 1993 because these accounts are not shown with product detail.

Although these margins are non-zero for indivi- dual products, they add up to zero in total because the amount added to the domestic supply of goods comes from the domestic supply of distribution, insurance and transport services.
Margins are thus shown in Table 14.10 separately for margins on domestic production and imports (cif/fob adjustment), because the SUT displays product detail down the columns.

In the aggregate, of course, these adjustments for trade and transport mar- gins on domestic production and the cif/fob adjustment for imports cancel each other out.
14.56 The SUT is primarily a matrix of ﬂows of goods and services designed to highlight the relationship between production and consumption, not between institutional units per se.

For example, households may undertake production in unincorporated enterprises for which activity appears in the production for own ﬁnal use part of the SUT, but also consume goods and ser- vices, as represented in individual consumption.
The current production transactions of the establishments of all institutional units are grouped together and sum- marized in one part of the SUT, and the remaining transactions are summarized and organized in another part.

Each institutional sector, including households (S.14), has its own SUT in principle.

The SUT for the total economy (S.1) is the cell-by-cell sum of the insti- tutional sector SUTs.
The consumer price index among major price indices 14.57 It is instructive at this point to associate the SUT with the component aggregates and matrices of the four major, headline price indices that are compiled by most countries.

In so doing, we form a more precise impression of the central purpose of the major price indices in the overall economic statistical system repre- sented by the SNA 1993.
The four main price indices and their associated national accounts aggregates and matrices in the SUT are: – producer price index (PPI): output of resident pro- ducers (P.1); – consumer price index (CPI): ﬁnal consumption of households (P.31) for CPI reference aggregate #1, plus 252 gross ﬁxed capital formation of households (P.51) for CPI reference aggregate #2; – export price index (XPI): exports (P.6); – import price index (MPI): imports (P.7).
14.58 The location and coverage of these major price indicators as they directly apply to goods and services value aggregates in the national accounts are shown dia- grammatically in Table 14.11.

Chapter 15 characterizes a price index as a function of price relatives and weights, noting that, other than the formula for the index itself, the requisite features of the relatives and weights would be determined by the value aggregate.
These factors are: – what items to include in the index; – how to determine the item prices; – what transactions that involve these items to include in the index; – from what source to draw the weights used in the selected index formula.

Based on our survey of the goods and services accounts of the SNA 1993 culminating in the SUT, these parti- culars for each of the four major indices can summarized as in Table 14.12.
Scope of the expenditure aggregates of the consumer price index 14.59 As noted in paragraphs 14.6 and 14.7, there are two principal expenditure sub-aggregates of the total ﬁnal expenditure of the households (S.14) institutional sector employed in most national CPIs that we can now see are transparently linked to the SNA: CPI reference aggregate #1, comprising the consump- tion items: P.311 Monetary consumption expenditure (Table 14.5) P.313 Financial measured (FISIM) (Table 14.5) P.312 Imputed expenditure on owner-occupied hous- ing services (Table 14.5) intermediation services implicitly CPI reference aggregate #2, comprising the consump- implicitly intermediation services tion and capital formation items: P.311 Monetary consumption expenditure (Table 14.5) P.313 Financial measured (FISIM) (Table 14.5) P.511a Gross ﬁxed capital formation in residential structures (Table 14.9) 14.60 Proponents of CPI reference aggregate #1 generally take a consumption or cost of living view of the CPI, seeing household welfare as determined by the ﬂow of goods and services, including the services of residential structures that are owned wholly or in part by the occupants, that households consume.
On this view, households’ ﬁxed capital formation, which is eﬀectively limited to the purchase of residences for own use, is a business-related activity of unincorporated enterprises that households own and thus not in the scope of the CPI.

The customary version of aggregate #1 excludes non-housing consumption from own production P.314.

Although compensation in kind in the form of beneﬁts s s e l s e x a T t n e m t s u d a j b o f / f i C b o f , s t r o p m I 7 .
, h c h w i f o , t u p t u O 1 .
: .

4 1 l e b a T s t r o p m i n o s e c i r p ’ - n o n r e h t O 3 1 .

P t u p t u o t e k r a m r o f t u p t u O 2 1 .

P e s u l a n ﬁ n w o t u p t u o t e k r a M 1 1 .
+ : t c u d o r p y b s e c r u o s e R s d o o G - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - 0 s e s u l a n F i : , ) s e c i r p i c s a b - n o n o t l d o s t a ( , .
P 3 5 .

 .

P s s o r G 1 5 .

P l a t i p a c d e x ﬁ n o i t a m r o f 5 1 .

S 1 s H S P N I l i a c o S 6 3 .

D n i s r e f s n a r t d n k i r e h t O 4 1 3 .

P d e t u p m i l i a u d v d n i i d e t u p m I 2 1 3 .

P - .

 .
.

S 3 1 .

S 3 1 .

S l s d o h e s u o H 3 9 9 1 A N S 4 1 .
# e t a g e r g g a e c n e r e f e r I P C 2 # : t c u d o r p - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - b o f , s t r o p x E 6 .
P n o i t a m r o f l a t i p a c s s o r G 5 .

P n o i t p m u s n o c e v i t c e l l o C 2 3 .

P n o i t p m u s n o c l i a u d v d n I i 1 3 .

P 3 9 9 1 A N S n o i t c a s n a r t .

 .
- n o N 1 i s e c v r e S : CONSUMER PRICE INDEX MANUAL: THEORY AND PRACTICE Table 14.12 Deﬁnition of scope, price relatives, coverage and weights for major price indices Index Items to include Price determination Transactions coverage Sources of weights PPI All types of domestically produced or processed goods and services that are valued at market prices Basic prices, determined for goods as the date when available for sale (available for change of ownership) or service price when service rendered Output of resident enterprises, comprising sales plus change in ﬁnished goods inventories for goods, and sales for services The product by industry matrices of Market output P.11 and Output for own ﬁnal use P.12 in the expanded Industry production account and the supply and use table (SUT) Reference expenditure aggregate #1: Reference expenditure aggregate #1: CPI Reference expenditure aggregate #1: All types of goods and services purchased explicitly or implicitly by households for individual consumption Purchasers’ prices, determined for goods on the change of ownership date and for services when used, including taxes on products, excluding subsidies on products, and including transport and distribution margins Reference expenditure aggregate #2 : All types of goods and services purchased explicitly by households for individual consumption, plus all types of goods and services
 purchased explicitly by households for residential capital formation Consumption expenditures of the Households sector S.13 of institutional units, excluding consumption from own production except for imputed expenditures for rental of owner-occupied dwellings Reference expenditure aggregate #2 : Reference expenditure aggregate #1, less imputed expenditures for rental of owner- occupied dwellings, plus net acquisition of or major improvements in residential housing The product column of the CPI consumption sub-aggregate of Individual consumption P.31 of the Household sector S.13 in the expanded Use of Income account and in the SUT Reference expenditure aggregate #2 : The product column of the monetary consumption sub-aggregate of individual consumption P.31 of the Household sector S.13 in the expanded Use of income account plus the product column of acquisitions less disposals of ﬁxed assets P.511 for residential housing The product column of Exports P.6 in the expanded External account of goods and services and the SUT XPI All types of transportable goods and services purchased by non-residents from residents.
Goods exported without change of ownership for signiﬁcant processing by non-residents and subsequent re-import are included Purchasers’ prices at the national frontier of the exporting country (fob), including export taxes and excluding export subsidies, and including transport and distribution margins from the production location to the national frontier All transportable goods and services produced or processed by residents and purchased by non-residents except goods in transit or goods exported and minimally processed by non-residents for re-import MPI All types of transportable goods and services purchased by residents from non-residents.
Goods imported without change of ownership for signiﬁcant processing by residents and subsequent re-export are included Basic prices at the national frontier of the exporting country (fob), excluding import taxes and including import subsidies, and excluding transport and distribution margins from the production location to the national frontier1 All transportable goods and services produced or processed by non- residents and purchased by residents except goods in transit or goods imported and minimally processed by residents for re-export The product column of Imports P.7 in the expanded External account of goods and services and the SUT PPI = producer price index; CPI = consumer price index; XPI = export price index; MPI = import price index.
1In deﬁning the import price index, however, the price index maker would, in fact, ﬁrst consider an economic input price index valuing imported goods and services at the purchasers’ price payable by their ﬁrst resident owner.

The import price index would be obtained by adjusting (multiplying) the import purchasers’ price index by a ‘‘markdown’’ index tracking the movement in the ratio of imports fob to imports at purchasers’ prices.
This is required for it to be properly matched in valuation with imports fob and yield the conceptually correct import volume index when used as an imports fob deﬂator.

254 provided by the employer is an important part of this item, households often are only vaguely aware of its value, since the employer actually makes the payments to the providers of the beneﬁts.
An argument nevertheless could be made for including this item, as households sometimes are able to exercise control over how this part of their compensation income is spent.
14.61 Proponents of CPI reference aggregate #2 generally take a transactions or inﬂation view of the CPI, tailoring the index to measuring the rate of change in the prices of an expenditure aggregate broadly covering the monetary ﬁnal expenditures that households make on goods and services, including their capital formation in residential structures via purchase of their own dwellings and the major improvements they make to them.

14.62 Both CPI concepts are useful.


The cost of liv- ing view provides a price index whose dual is the volume of household consumption.

The inﬂation view provides a price index whose dual is the volume of households’ ﬁnal monetary purchases, which represent the demand pres- sure they put on the markets in which they participate.

Table 14.11 illustrates the coverage of both indices.
The consumer price index as a measure of inﬂation in market transactions 14.63 Central banks take an interest in the major price indices, particularly if they are implementing a monetary policy that targets inﬂation.

Indeed, reference aggregate #2 has been seen as a better measure of change in the prices of actual transactions in goods and services than CPIs based on reference aggregate #1, which gives substantial weight to the imputed rent of owner-occupied housing.
14.64 Both reference aggregates for the CPI are an important component of total ﬁnal expenditure and GDP in virtually all countries, but the total value of transactions in goods and services also includes inter- mediate consumption, so as an inﬂation index for total goods and services transactions, the CPI’s coverage is rather limited under either deﬁnition #1 or #2 compared with, for example, the PPI, which covers, in principle, total output.
Progress in extending the industry coverage of the PPI to cover all output-producing activities, ser- vices in particular, has, however, proceeded slowly owing to the technical diﬃculty of specifying service products and measuring the associated prices.

The combination of the PPI, covering output, and the import price index provides a price index for total market supply, and is seen by at least one monetary authority as a useful inﬂation measure.
Another central bank targets the total domestic supply price index, which is based on total supply less exports (that is, covering the aggregate comprising output plus imports minus exports).

14.65 The CPI’s purchasers’ price valuation princi- ple also includes taxes less subsidies on products, which may not be desired in an inﬂation indicator for under- lying price change.
Nevertheless, the CPI is the most widely available macroeconomic price statistic, and may in many countries be the only available option for inﬂa- tion measurement.

Monetary authorities also may ﬁnd the CPI the most socially acceptable inﬂation target precisely because of its focus on households.
THE SYSTEM OF PRICE STATISTICS Treatment of cross-border shopping in the consumer price index 14.66 Exports P.6 are not an expenditure of any resident institutional unit and thus would not be the focus of a price index covering its expenditure.

By impli- cation, they would not appear in any CPI expenditure aggregate.

Imports are, however, an expenditure of res- ident units and it is often relevant to consider the impor- tance of imports in the expenditure aggregates of such units.
In many countries, imports acquired by households directly through cross-border shopping are a signiﬁcant fraction of household consumption expenditure.

14.67 Of particular note here is that imported goods P.71 and services P.72 in Table 14.8 for the household sector would contain only the direct expenditures of households on goods and services secured from non- residents, that is, in cross-border shopping.
This should include purchases of transportable goods and services by households from non-resident suppliers through all means, including in person, by mail order and through the Internet.
These expenditures in transactions with non-residents are already covered in households’ indivi- dual consumption P.31 and capital formation P.5, so the purpose of identifying imports P.7 in the context of the CPI is to identify the importance of transactions with non-residents in the ﬁnal expenditure aggregates of households and that part of those aggregates covered by the CPI expenditure aggregate.
14.68 Note that under both CPI reference aggregates #1 and #2 we would include expenditures on consump- tion goods and services provided by non-residents to resident households as the imported component of Individual consumption P.31.
To assess the importance of imports when considering CPI reference expenditure aggregate #2, we also would include households’ Fixed capital formation P.51 expenditures on imported trans- portable goods such as building materials for residences, as well as residential construction services provided by non-residents.
Other price indicators in the national accounts Price indices for total supply 14.69 Consistent with our earlier discussion of the coverage of the PPI, we deﬁne total market-valued out- put as the sum of market output P.11 and output for own ﬁnal use P.12.

Total output P.1 is the sum of market- valued output and other non-market output P.13.

Total supply at basic prices is the sum of output and imports P.7.
Mark-up adjustments at the product level for trade and transport margins on domestic production, insur- ance and freight on imports, and taxes D.21 less subsidies D.31 on products would be added to total supply at basic prices to produce total supply at purchasers’ prices.

14.70 In decomposing total supply into price and volume components, the total supply price index (SPI) at basic prices can be seen to be a weighted mean of the total output price index YPI and the import price index MPI.
The YPI comprises in turn the PPI and an implicit 255 CONSUMER PRICE INDEX MANUAL: THEORY AND PRACTICE deﬂator index (IDI) for other non-market output.

To obtain the price index for total supply at purchasers’ prices, the SPI would be multiplied by an index of the total mark-up for trade, insurance, and transport mar- gins, and taxes net of subsidies on products.

The margins only matter when developing supply price indices at purchasers’ prices for individual products and product sub-aggregates.
For all products, they cancel out, leaving only taxes less subsides on products contributing to the total mark-up on total supply at basic prices.

Total supply price indices at product levels of detail are useful in compiling and reconciling discrepancies in supply and use tables expressed in volume terms.
In addition, they are employed in producing industry price indices for intermediate consumption P.2, which are useful for compiling gross domestic product (GDP) volume mea- sures from the production approach.
Although princi- pally used as a compilation aid and in deﬂation of value added at basic prices via the double deﬂation approach (see paragraphs 14.71 and 14.73), supply price indices could also serve as analytical indicators in their own right because of their coverage of all goods and services transactions in the economy relating to production and external trade.
As such, they may be useful as indicators for the analysis and evaluation of economic policy, where broad coverage of transactions is required, for example in formulating monetary policy.
Price indices for intermediate consumption 14.71 In considering intermediate consumption price indices (IPIs) for the total economy and for industry, the weights correspond to a column-wise reading of the intermediate consumption part of the SUT’s use matrix, which is derived from Table 14.2 and shown in Table 14.10 as the region labelled P.2.
Because the various margins on basic prices inherent in prevailing pur- chasers’ prices may vary from user industry to user industry, the ideal sources for purchasers’ prices for intermediate consumption price indices would be enter- prise surveys.

Unfortunately, such surveys are generally burdensome and expensive.
Instead, as noted in the discussion above on price indices for total supply, the price index of intermediate consumption by industry can be derived from detailed product components of the SPI, which will result in indices of acceptable accuracy if the variation in the total tax, subsidy, transport and dis- tribution margin is not too great from industry to industry within product class.
For the total economy, the price index of intermediate consumption is obtained as a weighted average of industries’ intermediate input price indices, where the weights are the share of each industry’s intermediate consumption in the total intermediate consumption in the economy.
Price indices for ﬁnal uses 14.72 The price indices for ﬁnal use comprise deﬂa- tors for individual consumption P.31, collective con- sumption P.32, gross ﬁxed capital formation P.51, change in inventories P.52, acquisitions less disposals of valuables P.53 and exports P.6.
Of the major price indices discussed above, the CPI is the principal source of detailed (product level) information for P.31, while the PPI is a signiﬁcant source of detailed information for P.51 and the principal source for the ﬁnished goods component of P.52.

When the CPI is deﬁned on the basis of CPI reference expenditure aggregate #2, the CPI could also be the source of data on capital formation in residential structures.
The SPI may be the principal source for the input inventories component of P.52 in the absence of a detailed survey of the purchase price of intermediate inputs, and the XPI is the deﬂator for P.6.

The SPI can serve, as well, as a source of detailed product information for P.32, P.51 and P.53.

We will designate the deﬂator for total ﬁnal uses as the ﬁnal uses price index (FPI), which would be computed as a weighted mean (formula to be determined) of the component indices just discussed.
Price indices for gross domestic product the 14.73 As noted above in the discussion of SPI and the intermediate consumption price index, the Table 14.13 Generation of income account for establishment, institutional unit or institutional sector SNA 1993 goods and services items shown in bold Resources B.1 Value added 1 Uses D.1 Compensation of employees D.11 Wages and salaries D.12 Employers’ social contributions D.121 Employers’ actual social contributions D.122 Employers’ imputed social contributions D.2 Taxes on production and imports D.29 Other taxes on production2 D.3 Subsidies D.39 Other subsidies on production ()3 B.2 Operating surplus4 1From the production account.
2Taxes on production unrelated to products.

3Subsidies on production unrelated to products.

4Balancing item of the generation of income account.

256 that GDP price index can be compiled in two ways, corre- sponding to the two goods and services methods of compiling GDP: the production approach and the expenditure approach.
Recall the production approach derives from the deﬁnition of value added implicit in equation (14.2), as the diﬀerence between output P.1 (at basic prices) and intermediate con- sumption P.2 (at purchasers’ prices).

The SNA 1993 recommends the use of double deﬂation for value added, by which output at basic prices Y is deﬂated by all the items YPI to obtain output volume, and intermediate purchases are deﬂated by an inter- mediate purchases price index to obtain intermediate input volume.
Real value added is then computed as the diﬀerence between output volume and intermediate input volume (see SNA 1993, Chapter XVI).

This operation is equivalent to deﬂating value added in current prices with a double deﬂation-type price index having a positive weight on the YPI and a nega- tive weight on the IPI.
In the usual case just des- cribed, we have the value added deﬂator as a Paasche index of the output price index YPI s,t and the inter- mediate input price index IPI s,t, where the weight on the IPI s,t is wt I = P:2t P:1t P:2t is 1 wt and the weight on the YPI s,t I .
The corre- sponding volume index has the Laspeyres or ‘‘constant price’’ form, which is equivalent to the double deﬂation measure of the volume of real value added divided by current price value added in period s. The total value added at current basic prices divided by real value added, obtained via double deﬂation, yields the implicit deﬂator for value added at basic prices.
Finally, the GDP deﬂator at purchasers’ prices is the value added price index (at basic prices for output and purchasers’ prices for inter- mediate input) multiplied by the index of the mark-up on value added of output taxes less output subsidies on products.

14.74 Alternatively, the ﬁnal expenditure deﬂator FPI may be combined with the MPI using a double deﬂation-type approach.
GDP volume is calculated from expenditure data by deﬂating imports P.7 by the MPI, and subtracting the result from the volume of ﬁnal uses, calculated by deﬂating ﬁnal uses by the FPI.

The implicit GDP deﬂator would be the ratio of GDP at current prices to GDP volume so calculated.

The aggregate index of GDP volume and the aggregate index of real value added should agree with one another, as should, by implication, the implicit GDP deﬂator calculated from the two approaches.
Price indices for labour services 14.75 Value added appears ﬁrst in the production account, calculated as the balancing item between out- put and intermediate consumption.

This margin is used to pay for, among other things, labour services.

The SNA 1993 provides the income components comprising value added in the generation of income for THE SYSTEM OF PRICE STATISTICS account, shown in Table 14.13.
The largest of the income components itemized in this account is com- pensation of employees D.1, comprising wages and salaries D.11 and employers’ social contributions D.12.

D.1 represents a value aggregate for a ﬂow of labour services and is thus susceptible to decomposition into price and volume components.

Table 14.14 shows the same account expanded by type of labour service (occupation) for an establishment or industry.
The price index for labour services (LPI ) measures devel- opments in total compensation, by occupation, within industry.

The price of labour services in total compen- sation terms is of particular interest when compared with the GDP deﬂator, which indicates the relative purchasing power of labour compensation in terms of production for ﬁnal consumption.

This comparison is useful in assessing cost-push pressures on output prices and as an input into compiling measures of the pro- ductivity of labour.
A second useful comparison is between the wages and salaries sub-index of the LPI and the CPI.

The ratio of the LPI to the CPI indicates the purchasing power of wages in terms of consumption goods and services, and tracks the material welfare, particularly of the employees subsector S.143 of the household institutional sector S.14 (see Box 14.1 on page 238).
In the LPI, the price of labour services comprises all the components of compensation of employees, including employers’ social contributions (beneﬁts), as well as wages and salaries.

The wages and salaries sub-index of the LPI would be another example of a price index adjusted by a mark-up index.
Ana- logously with the price index for total supply at purchasers’ prices or for GDP by production in Table 14.10, the LPI would be adjusted in this case by a ‘‘markdown index’’, deducting employers’ social con- tributions.
Framework for a system of price statistics for goods and services 14.76 To summarize this overview of the main price indicators and the national accounts, Table 14.15 shows in tabular form the price indices needed for the value aggregates in the national accounts and their relation to the four main price indicators.
Indices that are functions of two other indices are shown with the notation f (I1; I2; w) where f is an index formula, I1 and I2 are price indices, w is the weight of the second index, with the weight of the ﬁrst index understood to be 1w. For example, if f is the Laspeyres formula, then the output price index (YPI) would be calculated by making the fol- lowing substitutions: Ps;t L =YPI s;t 1= 1 ws 2=ws D. f could also be chosen as a Paasche formula (with the same substitutions except for change in the time superscript on the weights 1=1 wt wt D), Fisher ideal formula, or other index formula.
rs;t 1 =PPI s;t, ws 2 =IDI s;t, ws D and wt 2= wt D, rs;t 257 CONSUMER PRICE INDEX MANUAL: THEORY AND PRACTICE 2 d e d d a l e u a V 1 .

 .

D s t r o p m i d n a n o i t c u d o r p n o s e x a T 2 .
, s r o t a s g e L i l l i s a n o s s e f o r P : 1 : 2 : 3 : 4 : 5 : 6 : 7 : 8 : 9 : , : 1 : 2 : 3 : 4 : 5 : 6 : 7 : 8 : 9 : a
, : 1 : 2 : 3 : 4 : 5 : 6 : 7 : 8 : 9 : 0 s e e y o p m e l f o n o i t a s n e p m o C 1 .
D s n o i t u b i r t n o c l a i c o s ’ s r e y o p m E l 2 1 .

D s e i r a l a s d n a s e g a W 1 1 .

D n n n n n .

S : : D I t i n u l a n o i t u t i t s n I a a a a : ) I C S I ( e d o c y r t s u d n I / y t i v i t c A n 1 .
P : s u t a t s t e k r a M e e e e e e e e : ) 1 l a n o i t a p u c c o ( .

4 1 l e b a T 258 ) ( i i s e d s b u S 3 .

D n o i t c u d o r p n o i i s e d s b u s r e h t O 9 3 .

D l 3 s u p r u s g n i t a r e p O 2 .

B .


 .

t n u o c c a n o i t c u d o r p e h t m o r F 2 . )
0 9 9 1 , a v e n e G ( ) 8 8 - O C S I ( : O L I f o s p u o r g j r o a m g n w o h S 1 i THE SYSTEM OF PRICE STATISTICS International comparisons of expenditure on goods and services 14.77 The main price statistics discussed thus far trace price developments of goods and services through time.
Purchasing power parities (PPPs) compare price levels between diﬀerent countries or geographical areas for a given accounting period and are generally used to eliminate the eﬀect of prices in diﬀerent currency units when comparing the levels of GDP between two coun- tries or areas.

The price relatives in bilateral PPPs com- prise the ratios of the local currency prices of identical goods and services between the two countries or areas.
The weights are proportional to the shares of these items in expenditure on GDP within the two countries or areas.

The sources of price relatives are the same as those for the ﬁnal uses GDP deﬂator, and the weights are simply the total ﬁnal uses, net of imports fob, by product.

In order to ensure that the PPP between area A and area B is the reciprocal of the PPP between B and A, bilateral PPPs need to be computed using symmetric index num- bers such as the Fisher.
14.78 A matrix of bilateral PPPs provides a means of making not only direct bilateral comparisons, but also bilateral comparisons between any two areas as the product of a sequence of bilateral PPPs through any set of intervening areas, beginning with the ﬁrst area and ending with the second.
In order to ensure the consistency of such multilateral comparisons – for example, that a chain beginning with a given area and ending with the same area produces a PPP of unity – bilateral PPPs are adjusted to produce a transitive set of comparisons.
14.79 The four main index series dealt with in this chapter are related to PPPs because the prices collected for the CPI, PPI, XPI and MPI, in addition to their use in these temporal indices and in the temporal GDP price index, can also be used in international comparisons of expenditures on consumption, capital formation and trade.

See Annex 4 on the International Comparison Program for further details on PPPs.
259 : 6 P : 6 P 7 P + 1 P : : ¼ x w , : 1 P : 6 P 7 P + 1 P : : : : 1 P ¼ m w , ) m w ; I D I , I P P ( f ¼ I P Y : 1 P : 7 P + 1 P : ¼ y w , ) y w ; I P Y , I P M ( f ¼ I P S e c i r p r e c u d o r P ) I P P ( x e d n i r o t a ﬂ e d t i c i l p m I t e k r a m - n o n ) I D I ( t u p t u o r e h t o r o f e c i r p t u p t u O ) I P Y ( , ) I P M ( e c i r p ’ ’ s r e s a h c r u p b o f / n a y b n w o d k r a m e c i r p e c i r p l y p p u S ) I P S ( , l i a t e d t c u d o r p 1 .


 , l i a t e d t c u d o r p 1 .

 , l i a t e d t c u d o r p h t i w 1 .

 , e b a t e s u d n a l y p p u S 1 .
S y m o n o c e l a t o t , ) x w , y w ; I P X , I P Y , I P M ( f ¼ I P S ) I P S D ( x e d n i e c i r p l y p p u s c i t s e m o D 1 .
S y m o n o c e l a t o t l , e b a t e s u d n a l y p p u S , ) n o i t c u d o r p f o t s o c ( y r t s u d n i y b t c u d o r p s e c i r p i c s a B , ( s e c i r p c i s a B i g n d u l c n i , , ) s t n e d i s e r - n o n t c u d o r p y b y b y b , s e c i r p i c s a B t c u d o r p ’ s r e s a h c r u p ; ) 7 .
P d n a 1 .

P ( t c u d o r p y b , s e c i r p i c s a B , b o f s t r o p x e 6 .

P ( s e c i r p ) w o e b l y r t n e ’ ’ s e s U ‘ ‘ .
4 1 l e b a T 260 y b t c u d o r p , s e c i r p c i s a B y r t s u d n i 2 1 .

P + 1 1 .

P d e u l a v - t e k r a M t u p t u o 3 1 .

P + 2 1 .

P + 1 1 .

P = 1 .

P t u p t u o l a t o T 7 .

P s t r o p m I 6 .

P 7 .

P + 1 .

P 7 .

P + 1 .


P l , , e d a r t c i t s e m o D d n a , .
P t e k r a m - , t n u o c c a e m o c n i 3 1 .

 : s 7 P + s 1 P : l a t o t l e v e l - t c u d o r P . )
e t a g e r g g a e h t n i ( o s a l l d u o w s e c d n i i p u - k r a m t u p t u o t : 1 3 D t : 1 2 D + t : 7 P + t : 1 P t : 7 P + t : 1 P : s 1 3 D : s 1 2 D + s 7 P + s 1 P : : ¼ I M S p u - k r a m y p p u S l ) I M S ( x e d n i e s u d n a l y p p u S 1 .
S y m o n o c e l a t o t l , e b a t i s e c v r e s r o f , , s r e i t n o r f l , e b a t e s u d n a l y p p u S 1 .
S y m o n o c e l a t o t i s e c v r e s r o f ( , , ) r e d v o r p i t c u d o r p y b l , e b a y a P 1 3 .

D 1 2 .

D s s e l s e x a T s e c i r p ’ s r e s a h c r u P + 7 .

P + 2 1 .

P + 1 1 .

P 1 3 .

D 1 2 .
D s e c i r p ’ s r e s a h c r u p l , , - t c u d o r p I P P d n a I P C e h t m o r f l e v e l - ’ ) I P I ( x e d n i ) I P H ( , .
, l i a t e d t c u d o r p e s U h t i w 1 .

 , s e c i r p ’ s r e s a h c r u P , s e c i r p ’ s r e s a h c r u P t c u d o r p y b 2 .

P i e t a d e m r e t n I n o i t p m u s n o c 1 3 .
P n o i t p m u s n o c l i a u d v d n I i d n a 3 ] X w , V w , N w , K w , G w [ ¼ ~ww , , , : 2 3 P : : : 6 P + 5 P + 4 P + 3 P : : 1 5 P : : : 6 P + 5 P + 4 P + 3 P : : 2 5 P : : : 6 P + 5 P + 4 P + 3 P : : 3 5 P : : : 6 P + 5 P + 4 P + 3 P : : : 6 P : : 6 P + 5 P + 4 P + 3 P : : ) I P C ( x e d n i s e c i d n i - , 4 1 .
S r o t c e s l d o h e s u o h , l i a t e d t c u d o r p h t i w 1 3 .
P f o n o i t a c ﬁ i s s a l c - ) I P G ( ) I P K ( x e d n i e c i r p y r o t n e v n I ) I P N ( ) I P V ( x e d n i e c i r p t r o p x E ) I P X ( x e d n i t n u o c c a e m o c n i f o , .
, l i a t e d t c u d o r p 1 .

 , l i a t e d t c u d o r p 1 .

S y m o n o c e , l i a t e d t c u d o r p h t i w 1 .

 , l i a t e d t c u d o r p h t i w 1 .
, s e c i r p ’ s r e s a h c r u P t c u d o r p y b , s e c i r p ’ s r e s a h c r u P t c u d o r p y b , s e c i r p ’ s r e s a h c r u P t c u d o r p y b , s e c i r p ’ s r e s a h c r u P t c u d o r p y b t c u d o r p y b , ) r e i t n o r f c i t s e m o d b o f ( s e c i r p ’ , - e m o h f o t n e r 2 3 .
P 1 5 .

P 2 5 .

P 3 5 .

P 6 .

P , s e c i r p ’ s r e s a h c r u P d e t u p m i t p e c x e , 1 3 .

P ) ~ww , I P X , I P V , I P N , I P K , I P G , I P H ( f 4 e r e h w ¼ I P F e c i r p s e s u l a t o T ) I P F ( x e d n i 1 .

S y m o n o c e l a t o t l , e b a t e s u d n a l y p p u S , s e c i r p ’ s r e s a h c r u P t c u d o r p y b 6 .

P + 5 .

P + 3 .

P l d o h e s u o H 4 1 .
261 : s 1 3 D : s 1 2 D + s 2 P : t : 2 P t : 1 P s 1 P : ¼ * I M S s t c u d o r p n o i i s e d s b u s f o t e n ) I w ; I P I , I P S ( f * I M S ¼ , ) M w ; I P M , I P F ( f ¼ r o t a l f e d P D G t : 1 3 D t : 1 2 D + t : 2 P t : 1 P : 7 P P D G ¼ M w : , e b a t e s u d n a l y p p u S 1 .
, - k r a m a y b + 2 .

P 1 .

P = P D G 1 3 .

D 1 2 .

D + 5 .

P + 3 .

P = P D G r o , 7 .


P 6 .

P c i t s e m o d s s o r G t c u d o r p : s 2 P s 1 P : - k r a m d e d d a e u a v l l e v e l - y r t s u d n I l a t o t e h t e d u c n l i l d u o w * I M S s e c d n i i ) e t a g e r g g a e h t n i ( .
S y m o n o c e l a t o t , .

 .

 .

s t c u d o r p n o i i s e d s b u s = 1 3 .

D d n a , s t c u d o r p n o s e x a t = 1 2 .

D , e s u l a n ﬁ n w o r o f t u p t u o = 2 1 .

P , t u p t u o t e k r a m = 1 1 .


P 1 t o n n a c x e d n i e c i r p A .

 . )
s H S P N I ( - ) .
d t n o c ( .

4 1 l e b a T 262 .

P t u p t u o t e k r a m - n o n r e h t o r o f r o t a ﬂ e d t i c i l p m i e h T .
t u p t u o t e k r a m - , .
t u p t u o t e k r a m - , 1 3 .
P n o i t p m u s n o c l i a u d v d n i i f o e r e h , m e t i t s r ﬁ e h t f o i t h g e w e h t h t i w , ’ ’ e p y t - n o i t a ﬂ e d e b u o d l ‘ ‘ , n a g A .
.

 , s t h g e w i . )

E n o i t c e S , I V X r e t p a h C , 3 9 9 1 A N S e e s ( x e d n i e k i l n U 4 BASIC INDEX NUMBER THEORY 15 Introduction The answer to the question what is the Mean of a given set of magnitudes cannot in general be found, unless there is given also the object for the sake of which a mean value is required.
There are as many kinds of average as there are purposes; and we may almost say in the matter of prices as many purposes as writers.

Hence much vain controversy between persons who are literally at cross purposes.

(Edgeworth (1888, p. 347)).

15.1 The number of physically distinct goods and unique types of services that consumers can purchase is in the millions.

On the business or production side of the economy, there are even more commodities that are actively traded.
This is because ﬁrms not only produce commodities for ﬁnal consumption, but they also produce exports and intermediate commodities that are demanded by other producers.

Firms collectively also use millions of imported goods and services, thousands of diﬀerent types of labour services and hundreds of thousands of speciﬁc types of capital.
If we further distinguish physical com- modities by their geographical location or by the season or time of day that they are produced or consumed, then there are billions of commodities that are traded within each year in any advanced economy.

For many purposes, it is necessary to summarize this vast amount of price and quantity information into a much smaller set of numbers.
The question that this chapter addresses is: how exactly should the microeconomic information involving possi- bly millions of prices and quantities be aggregated into a smaller number of price and quantity variables?

This is the basic problem of index numbers.


15.2 It is possible to pose the index number problem in the context of microeconomic theory; i.e., given that we wish to implement some economic model based on producer or consumer theory, what is the ‘‘best’’ method for constructing a set of aggregates for the model?

When constructing aggregate prices or quantities, however, other points of view (that do not rely on economics) are possible.
Some of these alternative points of view are considered in this chapter and the next.

Economic approaches are pursued in Chapters 17 and 18.

15.3 The index number problem can be framed as the problem of decomposing the value of a well-deﬁned set of transactions in a period of time into an aggregate price term times an aggregate quantity term.

It turns out that this approach to the index number problem does not lead to any useful solutions.
So, in paragraphs 15.7 to 15.17, the problem of decomposing a value ratio pertaining to two periods of time into a component that measures the overall change in prices between the two periods (this is the price index) times a term that measures the overall change in quantities between the two periods (this is the quantity index) is considered.
The simplest price index is a ﬁxed basket type index; i.e., ﬁxed amounts of the n quantities in the value aggregate are chosen and then the values of this ﬁxed basket of quantities at the prices of period 0 and at the prices of period 1 are calculated.

The ﬁxed basket price index is simply the ratio of these two values where the prices vary but the quantities are held ﬁxed.
Two natural choices for the ﬁxed basket are the quantities transacted in the base period, period 0, or the quantities transacted in the current period, period 1.

These two choices lead to the Laspeyres (1871) and Paasche (1874) price indices, respectively.

15.4 Unfortunately, the Paasche and Laspeyres measures of aggregate price change can diﬀer, sometimes substantially.
Thus in paragraphs 15.18 to 15.32, taking an average of these two indices to come up with a single measure of price change is considered.

In paragraphs 15.18 to 15.23, it is argued that the ‘‘best’’ average to take is the geometric mean, which is Irving Fisher’s (1922) ideal price index.

In paragraphs 15.24 to 15.32, instead of averaging the Paasche and Laspeyres measures of price change, taking an average of the two baskets is con- sidered.
This ﬁxed basket approach to index number the- ory leads to a price index advocated by Correa Moylan Walsh (1901; 1921a).

Other ﬁxed basket approaches are, however, also possible.

Instead of choosing the basket of period 0 or 1 (or an average of these two baskets), it is possible to choose a basket that pertains to an entirely diﬀerent period, say period b.
In fact, it is typical statis- tical agency practice to pick a basket that pertains to an entire year (or even two years) of transactions in a year prior to period 0, which is usually a month.

Indices of this type, where the weight reference period diﬀers from the price reference period, were originally proposed by Joseph Lowe (1823), and indices of this type are studied in paragraphs 15.64 to 15.84.
Such indices are also evaluated from the axiomatic perspective in Chapter 16 and from the economic perspective in Chapter 17.1 15.5 In paragraphs 15.65 to 15.75, another approach to the determination of the functional form or the formula for the price index is considered.

This approach is attrib- utable to the French economist Divisia (1926) and is based on the assumption that price and quantity data are available as continuous functions of time.
The theory of diﬀerentiation is used in order to decompose the rate of change of a continuous time value aggregate into two 1Although indices of this type do not appear in Chapter 19, where most of the index number formulae exhibited in Chapters 15–18 are illustrated using an artiﬁcial data set, indices where the weight refer- ence period diﬀers from the price reference period are illustrated numerically in Chapter 22, in which the problem of seasonal com- modities is discussed.
263 CONSUMER PRICE INDEX MANUAL: THEORY AND PRACTICE components that reﬂect aggregate price and quantity change.

Although the approach of Divisia oﬀers some insights,2 it does not oﬀer much guidance to statistical agencies in terms of leading to a deﬁnite choice of index number formula.
15.6 In paragraphs 15.76 to 15.97, the advantages and disadvantages of using a ﬁxed base period in the bilateral index number comparison are considered versus always comparing the current period with the previous period, which is called the chain system.

In the chain system, a link is an index number comparison of one period with the previous period.

These links are multiplied together in order to make comparisons over many periods.
The decomposition of value aggregates into price and quantity components The decomposition of value aggregates and the product test 15.7 A price index is a measure or function which summarizes the change in the prices of many commod- ities from one situation 0 (a time period or place) to another situation 1.

More speciﬁcally, for most practical purposes, a price index can be regarded as a weighted mean of the change in the relative prices of the com- modities under consideration in the two situations.
To determine a price index, it is necessary to know: which commodities or items to include in the index; how to determine the item prices; which transactions that involve these items to include in the index; how to determine the weights and from which sources these weights should be drawn; what formula or type of mean should be used to average the selected item relative prices.
All the above questions regarding the deﬁnition of a price index, except the last, can be answered by appealing to the deﬁnition of the value aggregate to which the price index refers.
A value aggregate V for a given collection of items and transactions is computed as: piqi (15:1) n V=P i=1 where pi represents the price of the ith item in national currency units, qi represents the corresponding quantity transacted in the time period under consideration and the subscript i identiﬁes the ith elementary item in the group of n items that make up the chosen value aggregate V. Included in this deﬁnition of a value aggregate is the speciﬁcation of the group of included commodities (which items to include) and of the economic agents engaging in transactions involving those commodities (which trans- actions to include), as well as principles of the valuation and time of recording that motivate the behaviour of the economic agents undertaking the transactions (determi- nation of prices).
The included elementary items, their 2 In particular, it can be used to justify the chain system of index numbers (discussed in paragraphs 15.86 to 15.97).

264 valuation (the pi), the eligibility of the transactions and the item weights (the qi) are all within the domain of deﬁnition of the value aggregate.
The precise determination of the pi and qi is discussed in more detail elsewhere in this manual, in particular in Chapter 5.3 15.8 The value aggregate V deﬁned by equation (15.1) refers to a certain set of transactions pertaining to a single (unspeciﬁed) time period.

Now the same value aggregate for two places or time periods, periods 0 and 1, is con- sidered.
For the sake of convenience, period 0 is called the base period and period 1 is called the current period and it is assumed that observations on the base period price and quantity vectors, p0 [ p0 n ] and q0 n] respectively, have been collected.4 The value [q0 aggregates in the base and current periods are deﬁned in the obvious way as: 1, .

.

.

, q0 1, .

.

.


, p0 n V 0 P i=1 i q0 p0 i ; V 1 P i q1 p1 i (15:2) n i=1 In the previous paragraph, a price index was deﬁned as a function or measure which summarizes the change in the prices of the n commodities in the value aggregate from situation 0 to situation 1.
In this paragraph, a price index P( p0, p1, q0, q1) along with the corresponding quantity index (or volume index) Q( p0, p1, q0, q1) is deﬁned to be two functions of the 4n variables p0, p1, q0, q1 (these variables describe the prices and quantities pertaining to the value aggregate for periods 0 and 1) where these two functions satisfy the following equation:5 V 1=V 0=P( p0, p1, q0, q1) Q( p0, p1, q0, q1) (15:3) n=p0 n. 1=p0 If there is only one item in the value aggregate, then the price index P should collapse down to the single price ratio, p1 1, and the quantity index Q should collapse down to the single quantity ratio, q1 1.
In the case of many items, the price index P is to be interpreted as some sort of weighted average of the individual price ratios, p1 1=q0 1=p0 1, .

.

.

, p1 15.9 Thus the ﬁrst approach to index number theory can be regarded as the problem of decomposing the change in a value aggregate, V1/V 0, into the product of a part that is attributable to price change, P( p0, p1, q0, q1), and a part that is attributable to quantity change, Q( p0, p1, q0, q1).
This approach to the determination of the price index is the approach that is taken in the national accounts, where a price index is used to deﬂate a value ratio in order to obtain an estimate of quantity change.

Thus, in this approach to index number theory, the primary use for the price index is as a deﬂator.
Note that once the functional form for the price index P( p0, p1, q0, q1) is known, then the corresponding 3 Ralph Turvey has noted that some values may be diﬃcult to decompose into unambiguous price and quantity components.

Examples of diﬃcult-to-decompose values are bank charges, gambling expenditures and life insurance payments.

4 Note that it is assumed that there are no new or disappearing com- modities in the value aggregates.
Approaches to the ‘‘new goods prob- lem’’ and the problem of accounting for quality change are discussed in Chapters 7, 8 and 21.

5 The ﬁrst person to suggest that the price and quantity indices should be jointly determined in order to satisfy equation (15.3) was Fisher (1911, p. 418).

Frisch (1930, p. 399) called equation (15.3) the product test.
quantity or volume index Q( p0, p1, q0, q1) is completely determined by P; i.e., rearranging equation (15.3): Q( p0, p1, q0, q1)= V 1=V 0 =P( p0, p1, q0, q1) (15:4) Conversely, if the functional form for the quantity index Q(p0, p1, q0, q1) is known, then the corresponding price index P(p0, p1, q0, q1) is completely determined by Q.
Thus using this deﬂation approach to index number theory, separate theories for the determination of the price and quantity indices are not required: if either P or Q is determined, then the other function is implicitly determined by the product test equation (15.4).

15.10 In the next section, two concrete choices for the price index P( p0, p1, q0, q1) are considered and the corresponding quantity indices Q( p0, p1, q0, q1) that result from using equation (15.4) are also calculated.
These are the two choices used most frequently by national income accountants.

The Laspeyres and Paasche indices BASIC INDEX NUMBER THEORY PL( p0, p1, q0, q1) PP( p0, p1, q0, q1) n P i=1 n P i=1 n P i=1 n P i=1 i q0 p1 i i q0 p0 i i q1 p1 i p0 i q1 i (15:5) (15:6) 15.13 The formulae (15.5) and (15.6) can be rewrit- ten in an alternative manner that is more useful for statistical agencies.
Deﬁne the period t expenditure share on commodity i as follows: i pt st iqt i n P j=1 jqt pt j for i=1, .

.

.

, n and t=0, 1 (15:7) Then the Laspeyres index (15.5) can be rewritten as follows:10 15.11 One of the simplest approaches to the deter- mination of the price index formula was described in great detail by Lowe (1823).
His approach to measuring the price change between periods 0 and 1 was to specify an approximate representative commodity basket,6 which is a quantity vector q:[q1, .

.

.
.

.

, qn] that is representative of purchases made during the two periods under con- sideration, and then calculate the level of prices in period 1 relative to period 0 as the ratio of the period 1 cost of the basket, Pn i qi, to the period 0 cost of the basket, Pn i qi: This ﬁxed basket approach to the determina- tion of the price index leaves open the question as to how exactly is the ﬁxed basket vector q to be chosen.
i=1p1 i=1p0 15.12 As time passed, economists and price statisti- cians demanded a little more precision with respect to the speciﬁcation of the basket vector q.

There are two natural choices for the reference basket: the base period com- modity vector q0 or the current period commodity vector q1.
These two choices lead to the Laspeyres (1871) price index7 PL deﬁned by equation (15.5) and the Paasche (1874) price index8 PP deﬁned by equation (15.6):9 6 Lowe (1823, Appendix, p. 95) suggested that the commodity basket vector q should be updated every ﬁve years.

Lowe indices are studied in more detail in paragraphs 15.45 to 15.85.

7 This index was actually introduced and justiﬁed by Drobisch (1871a, p. 147) slightly earlier than Laspeyres.
Laspeyres (1871, p. 305) in fact explicitly acknowledged that Drobisch showed him the way forward.

However, the contributions of Drobisch have been forgotten for the most part by later writers because Drobisch aggressively pushed for the ratio of two unit values as being the ‘‘best’’ index number formula.

While this formula has some excellent properties where all the n commodities being compared have the same unit of measurement, it is useless when, say, both goods and services are in the index basket.
8 Drobisch (1871b, p. 424) also appears to have been the ﬁrst to deﬁne explicitly and justify the Paasche price index formula, but he rejected this formula in favour of his preferred formula, the ratio of unit values, and so again he did not gain any credit for his early suggestion of the Paasche formula.

9 Note that PL( p0, p1, q0, q1) does not actually depend on q1 and PP ( p0, p1, q0, q1) does not actually depend on q0.
It does no harm to include these vectors, however, and the notation indicates that the reader is in the realm of bilateral index number theory; i.e., the prices and quantities for a value aggregate pertaining to two periods are being compared.

PL( p0, p1, q0, q1)= P n i=1 n =P i=1 n =P i=1 i q0 p1 i n P j=1 j q0 p0 j ( p1 i =p0 i ) p0 i q0 i ( p1 i =p0 i )s0 i n P j=1 j q0 p0 j (15:8) i =p0 using deﬁnitions (15.7).
The Laspeyres price index PL can thus be written as an arithmetic average of the n price ratios, p1 i , weighted by base period expenditure shares.

The Laspeyres formula (until very recently) has been widely used as the intellectual base for consumer price indices (CPIs) around the world.
To implement it, a statistical agency needs only to collect information on expenditure shares s0 n for the index domain of deﬁnition for the base period 0, and then collect information on item prices alone on an ongoing basis.

Thus the Laspeyres CPI can be produced on a timely basis without having quantity information for the current period.
15.14 The Paasche index can also be written in expenditure share and price ratio form as follows:11 PP( p0, p1, q0, q1)= ( 1 n P j=1 ) j q1 p1 j n P i=1 p0 i q1 i = ( n P i=1 1 ð p0 i =p1 i Þp1 i q1 i ) n P j=1 j q1 p1 j 10 This method of rewriting the Laspeyres index (or any ﬁxed basket index) as a share weighted arithmetic average of price ratios is attri- butable to Fisher (1897, p. 517) (1911, p. 397) (1922, p. 51) and Walsh (1901, p. 506; 1921a, p. 92).
11 This method of rewriting the Paasche index (or any ﬁxed basket index) as a share weighted harmonic average of the price ratios is attributable to Walsh (1901, p. 511; 1921a, p. 93) and Fisher (1911, p. 397–398).

265 CONSUMER PRICE INDEX MANUAL: THEORY AND PRACTICE 1 ð i =p0 p1 i Þ1s1 i = n P i=1 n = P i=1 i =p0 p1 i 1 s1 i 1 (15:9) using deﬁnitions (15.7).
The Paasche price index PP can thus be written as a harmonic average of the n item price ratios, p1 i , weighted by period 1 (current period) expenditure shares.12 The lack of information on current period quantities prevents statistical agencies from pro- ducing Paasche indices on a timely basis.
i =p0 15.15 The quantity index that corresponds to the Laspeyres price index using the product test in equation (15.3) is the Paasche quantity index; i.e., if P in equation (15.4) is replaced by PL deﬁned by equation (15.5), then the following quantity index is obtained: QP( p0, p1, q0, q1) n P i=1 n P i=1 i q1 p1 i i q0 p1 i (15:10) Note that QP is the value of the period 1 quantity vector valued at the period 1 prices, Pn i , divided by the (hypothetical) value of the period 0 quantity vector valued at the period 1 prices, Pn i .
Thus the period 0 and 1 quantity vectors are valued at the same set of prices, the current period prices, p1.
i=1p1 i=1p1 i q0 i q1 15.16 The quantity index that corresponds to the Paasche price index using the product test (15.3) is the Laspeyres quantity index; i.e., if P in equation (15.4) is replaced by PP deﬁned by equation (15.6), then the following quantity index is obtained: QL( p0, p1, q0, q1) n P i=1 n P i=1 i q1 p0 i i q0 p0 i (15:11) Note that QL is the (hypothetical) value of the period 1 quantity vector valued at the period 0 prices, Pn i q1 i , divided by the value of the period 0 quantity vector valued at the period 0 prices, Pn i .
Thus the period 0 and 1 quantity vectors are valued at the same set of prices, the base period prices, p0.

i=1p0 i=1p0 i q0 15.17 The problem with the Laspeyres and Paasche index number formulae is that, although they are equally plausible, in general they will give diﬀerent answers.
For most purposes, it is not satisfactory for the statistical agency to provide two answers to the question:13 What is the ‘‘best’’ overall summary measure of price change for the value aggregate over the two 12 Note that the derivation in the formula (15.9) shows how harmonic averages arise in index number theory in a very natural way.

13 In principle, instead of averaging the Paasche and Laspeyres indices, the statistical agency could think of providing both (the Paasche index on a delayed basis).
This suggestion would lead to a matrix of price comparisons between every pair of periods instead of a time series of comparisons.

Walsh (1901, p. 425) noted this possibility: ‘‘In fact, if we use such direct comparisons at all, we ought to use all possible ones.’’ 266 indices?

Under periods in question?

In the following section, we con- sider how ‘‘best’’ averages of these two estimates of price change can be constructed.
Before doing so, we ask: What is the ‘‘normal’’ relationship between the Paasche and Laspeyres ‘‘normal’’ economic conditions when the price ratios pertaining to the two situations under consideration are negatively correlated with the corresponding quantity ratios, it can be shown that the Laspeyres price index will be larger than the corresponding Paasche index.14 A precise statement of this result is presented in Appendix 15.1.15 The diver- gence between PL and PP suggests that if a single esti- mate for the price change between the two periods is required, then some sort of evenly weighted average of the Laspeyres and Paasche indices should be taken as the ﬁnal estimate of price change between periods 0 and 1.
As mentioned above, this strategy will be pursued in the following section.

It should, however, be kept in mind that statistical agencies will not usually have information on current expenditure weights, hence averages of Paasche and Laspeyres indices can be pro- duced only on a delayed basis (perhaps using national accounts information) or not at all.
Symmetric averages of ﬁxed basket price indices The Fisher index as an average of the Paasche and Laspeyres indices 15.18 As mentioned above, since the Paasche and Laspeyres price indices are equally plausible but often give diﬀerent estimates of the amount of aggregate price change between periods 0 and 1, it is useful to consider taking an evenly weighted average of these ﬁxed basket price indices as a single estimator of price change between the two periods.
Examples of such symmetric averages16 are the arithmetic mean, which leads to the 14 Peter Hill (1993, p. 383) summarized this inequality as follows: It can be shown that relationship (13) [i.e., that PL is greater than PP] holds whenever the price and quantity relatives (weighted by values) are negatively correlated.
Such negative correlation is to be expected for price takers who react to changes in relative prices by substituting goods and services that have become relatively less expensive for those that have become relatively more expensive.

In the vast majority of situations covered by index numbers, the price and quantity relatives turn out to be negatively correlated so that Laspeyres indices tend systematically to record greater increases than Paasche with the gap between them tending to widen with time.
15 There is another way to see why PP will often be less than PL.

If the period 0 expenditure shares s0 i are exactly equal to the corres- ponding period 1 expenditure shares s1 i , then by Schlo¨ milch’s (1858) inequality (see Hardy, Littlewood and Polya´ (1934, p. 26)), it can be shown that a weighted harmonic mean of n numbers is equal to or less than the corresponding arithmetic mean of the n numbers and the inequality is strict if the n numbers are not all equal.
If expenditure shares are approximately constant across periods, then it follows that PP will usually be less than PL under these conditions (see paragraphs 15.70 to 15.84).

16 For a discussion of the properties of symmetric averages, see Diewert (1993c).

Formally, an average mða, bÞ of two numbers a and b is symmetric if mða, bÞ ¼ mðb, aÞ.

In other words, the numbers a and b are treated in the same manner in the average.

An example of a non- symmetric average of a and b is (1/4)a+(3/4)b.
In general, Walsh (1901, p. 105) argued for a symmetric treatment if the two periods (or countries) under consideration were to be given equal importance.
Drobisch (1871b, p. 425), Sidgwick (1883, p. 68) and Bowley (1901, p. 227)17 index, PD (1=2)PL+(1=2)PP, and the geometric mean, which leads to the Fisher (1922)18 ideal index, PF, deﬁned as PF ( p0, p1, q0, q1) PL( p0, p1, q0, q1ÞPP( p0, p1, q0, q1) 1=2 (15:12) At this point, the ﬁxed basket approach to index number theory is transformed into the test approach to index number theory; i.e., in order to determine which of these ﬁxed basket indices or which averages of them might be ‘‘best’’, desirable criteria or tests or properties are needed for the price index.
This topic will be pursued in more detail in the next chapter, but an introduction to the test approach is provided in the present section because a test is used to determine which average of the Paasche and Laspeyres indices might be ‘‘best’’.

15.19 What is the ‘‘best’’ symmetric average of PL and PP to use as a point estimate for the theoretical cost of living index?
It is very desirable for a price index formula that depends on the price and quantity vectors pertaining to the two periods under consideration to satisfy the time reversal test.19 An index number formula P( p0, p1, q0, q1) satisﬁes this test if P( p1, p0, q1, q0)=1=P( p0, p1, q0, q1) (15:13) i.e., if the period 0 and period 1 price and quantity data are interchanged, and then the index number formula is evaluated, then this new index P( p1, p0, q1, q0) is equal to the reciprocal of the original index P( p0, p1, q0, q1).
This is a property that is satisﬁed by a single price ratio, and it seems desirable that the measure of aggregate price change should also satisfy this property so that it does not matter which period is chosen as the base period.
Put another way, the index number comparison between any two points of time should not depend on the choice of which period we regard as the base period: if the other period is chosen as the base period, then the new index number should simply equal the reciprocal of the original index.

It should be noted that the Laspeyres and Paasche price indices do not satisfy this time reversal property.
17 Walsh (1901, p. 99) also suggested the arithmetic mean index PD (see Diewert (1993a, p. 36) for additional references to the early history of index number theory).

18 Bowley (1899, p. 641) appears to have been the ﬁrst to suggest the use of the geometric mean index PF.

Walsh (1901, pp.
Walsh (1901, pp.

428–429) also suggested this index while commenting on the big diﬀerences between the Laspeyres and Paasche indices in one of his numerical examples: ‘‘The ﬁgures in columns (2) [Laspeyres] and (3) [Paasche] are, singly, extravagant and absurd.
But there is order in their extravagance; for the nearness of their means to the more truthful results shows that they straddle the true course, the one varying on the one side about as the other does on the other.’’ 19 See Diewert (1992a, p. 218) for early references to this test.

If we want the price index to have the same property as a single price ratio, then it is important to satisfy the time reversal test.

However, other points of view are possible.


For example, we may want to use our price index for compensation purposes, in which case satisfaction of the time reversal test may not be so important.
BASIC INDEX NUMBER THEORY 15.20 Having deﬁned what it means for a price index P to satisfy the time reversal test, then it is possible to establish the following result.20 The Fisher ideal price index deﬁned by equation (15.12) is the only index that is a homogeneous21 symmetric average of the Laspeyres and Paasche price indices, PL and PP, and satisﬁes the time reversal test (15.13).
The Fisher ideal price index thus emerges as perhaps the ‘‘best’’ evenly weighted average of the Paasche and Laspeyres price indices.
15.21 It is interesting to note that this symmetric basket approach to index number theory dates back to one of the early pioneers of index number theory, Arthur L. Bowley, as the following quotations indicate: If [the Paasche index] and [the Laspeyres index] lie close together there is no further diﬃculty; if they diﬀer by much they may be regarded as inferior and superior limits of the index number, which may be estimated as their arithmetic mean .

.

.

as a ﬁrst approximation (Bow- ley (1901, p. 227)).


When estimating the factor necessary for the correc- tion of a change found in money wages to obtain the change in real wages, statisticians have not been content to follow Method II only [to calculate a Laspeyres price index], but have worked the problem backwards [to calculate a Paasche price for- wards.

.

.

.
They have then taken the arithmetic, geo- metric or harmonic mean of the two numbers so found (Bowley (1919, p. 348)).22 15.22 The quantity index that corresponds to the Fisher price index using the product test (15.3) is the Fisher quantity index; i.e., if P in equation (15.4) is replaced by PF deﬁned by equation (15.12), the follow- ing quantity index is obtained: index] as well as QF ( p0, p1, q0, q1) QL( p0, p1, q0, q1)QP( p0, p1, q0, q1) 1=2 (15:14) Thus the Fisher quantity index is equal to the square root of the product of the Laspeyres and Paasche quantity indices.
It should also be noted that QF ( p0, p1, q0, q1)=PF (q0, q1, p0, p1); the role of prices i.e., and quantities is interchanged in the Fisher price index formula, then the Fisher quantity index is obtained.23 if 15.23 Rather than take a symmetric average of the two basic ﬁxed basket price indices pertaining to two situations, PL and PP, it is also possible to return to Lowe’s basic formulation and choose the basket vector q to be a symmetric average of the base and current period basket vectors, q0 and q1.
This approach 20 See Diewert (1997, p. 138).

21 An average or mean of two numbers a and b, m(a, b), is homogeneous if when both numbers a and b are multiplied by a positive number l, then the mean is also multiplied by l; i.e., m satisﬁes the following property: m(la, lb)=lm(a, b).

22 Fisher (1911, pp.

417–418; 1922) also considered the arithmetic, geometric and harmonic averages of the Paasche and Laspeyres indi- ces.
23 Fisher (1922, p. 72) said that P and Q satisﬁed the factor reversal test if Q( p0, p1, q0, q1)=P(q0, q1, p0, p1) and P and Q satisﬁed the pro- duct test (15.3) as well.

267 CONSUMER PRICE INDEX MANUAL: THEORY AND PRACTICE to index number theory is pursued in the following section.

15.26 It should be noted that Walsh (1901, p. 105; 1921a) also saw the price index number problem in the above framework: The Walsh index and the theory of the ‘‘pure’’ price index 1, p0 2, .

.

.
.

.

, p0 15.24 Price statisticians tend to be very comfortable with a concept of the price index that is based on pricing out a constant ‘‘representative’’ basket of commodities, q (q1, q2, .

.

.

, qn), at the prices of periods 0 and 1, p0 ( p0 n) and p1 ( p1 n) respec- tively.
Price statisticians refer to this type of index as a ﬁxed basket index or a pure price index 24 and it corre- sponds to Sir George H. Knibbs’s (1924, p. 43) unequivocal price index.25 Since Lowe (1823) was the ﬁrst person to describe systematically this type of index, it is referred to as a Lowe index.

Thus the general functional form for the Lowe price index is 2, .

.

.
.

.

, p1 1, p1 PLo( p0, p1, q) P n i=1 p1 i qi n P i=1 n i qi=P p0 i=1 si( p1 i =p0 i ) (15:15) where the (hypothetical) hybrid expenditure shares si corresponding to the quantity weights vector q are deﬁned by: 26 Commodities are to be weighted according to their importance, or their full values.

But the problem of axiometry always involves at least two periods.

There is a ﬁrst period, and there is a second period which is com- pared with it.
Price variations have taken place between the two, and these are to be averaged to get the amount of their variation as a whole.

But the weights of the com- modities at the second period are apt to be diﬀerent from their weights at the ﬁrst period.

Which weights, then, are the right ones – those of the ﬁrst period?

Or those of the second?

Or should there be a combination of the two sets?

There is no reason for preferring either the ﬁrst or the second.
Then the combination of both would seem to be the proper answer.

And this combination itself involves an averaging of the weights of the two periods (Walsh (1921a, p. 90)).

Walsh’s suggestion will be followed and thus the ith quantity weight, qi, is restricted to be an average or mean of the base period quantity q0 i and the current period quantity for commodity iq1 i , q1 i ), for i= 1, 2, .

.

.
.

.

, n.28 Under this assumption, the Lowe price index (15.15) becomes: i , say m(q0 si p0 i qi n P j=1 p0 j qj for i=1, 2, .

.

.

, n (15:16) PLo( p0, p1, q0, q1) n P i=1 n P j=1 i m(q0 p1 i , q1 i ) j m(q0 p0 j , q1 j ) (15:17) 15.25 The main reason why price statisticians might prefer a member of the family of Lowe or ﬁxed basket price indices deﬁned by equation (15.15) is that the ﬁxed basket concept is easy to explain to the public.
Note that the Laspeyres and Paasche indices are special cases of the pure price concept if we choose q=q0 (which leads to the Laspeyres index) or if we choose q=q1 (which leads to the Paasche index).27 The practical problem of picking q remains to be resolved, and that is the problem that will be addressed in this section.

24 See section 7 in Diewert (2001).


25 Suppose however that, for each commodity, Q0=Q, then the frac- tion, P (P0Q)= P (PQ), viz., the ratio of aggregate value for the second unit-period to the aggregate value for the ﬁrst unit-period is no longer merely a ratio of totals, it also shows unequivocally the eﬀect of the change in price.

Thus it is an unequivocal price index for the quantitatively unchanged complex of commodities, A, B, C, etc.
It is obvious that if the quantities were diﬀerent on the two occasions, and if at the same time the prices had been unchanged, the preceding formula would become P (PQ0)= P (PQ).

It would still be the ratio of the aggregate value for the second unit-period to the aggregate value for the ﬁrst unit period.

But it would be also more than this.

It would show in a generalized way the ratio of the quan- tities on the two occasions.
Thus it is an unequivocal quantity index for the complex of commodities, unchanged as to price and diﬀering only as to quantity.

Let it be noted that the mere algebraic form of these expressions shows at once the logic of the problem of ﬁnding these two indices is identical (Knibbs (1924, pp.

43–44)).

26 Note that Fisher (1922, p. 53) used the terminology ‘‘weighted by a hybrid value’’, while Walsh (1932, p. 657) used the term ‘‘hybrid weights’’.
27 Note that the ith share deﬁned by equation (15.16) in this case is the i =Pn hybrid share si p0 i , which uses the prices of period 0 and the quantities of period 1. i=1p0 i q1 i q1 268 15.27 In order to determine the functional form for the mean function m, it is necessary to impose some tests or axioms on the pure price index deﬁned by equation (15.17).

As above, we ask that PLo satisfy the time reversal test (15.13).
Under this hypothesis, it is imme- diately obvious that the mean function m must be a symmetric mean;29 i.e., m must satisfy the following property: m(a, b)=m(b, a) for all a>0 and b>0.

This assumption still does not pin down the functional form for the pure price index deﬁned by equation (15.17).
For example, the function m(a, b) could be the arithmetic mean, (1/2)a+(1/2)b, in which case equation (15.17) reduces to the Marshall (1887) and Edgeworth (1925) price index PME, which was the pure price index pre- ferred by Knibbs (1924, p. 56): PME( p0, p1, q0, q1) n P i=1 n P j=1 i f(q0 p1 i +q1 i )=2g (15:18) j f(q0 p0 j +q1 j )=2g 15.28 On the other hand, the function m(a, b) could be the geometric mean, (ab)1/2, in which case equation 28 Note that we have chosen the mean function m(q0 i ) to be the same for each item i.
We assume that m(a, b) has the following two prop- erties: m(a, b) is a positive and continuous function, deﬁned for all positive numbers a and b and m(a, a)=a for all a>0.

29 For more on symmetric means, see Diewert (1993c, p. 361).
i , q1 (15.17) reduces to the Walsh (1901, p. 398; 1921a, p. 97) price index, PW: 30 PW ( p0, p1, q0, q1) PW ( p0, p1, q0, q1) n P i=1 n P j=1 p ﬃﬃﬃﬃﬃﬃﬃﬃﬃ q0 i q1 i p1 i q ﬃﬃﬃﬃﬃﬃﬃﬃﬃ q0 j q1 j p0 j (15:19) 15.29 There are many other possibilities for the including the mean of order r, mean function m, [(1/2)ar+(1/2)br ]1/r for r 6¼ 0.
Obviously, in order to completely determine the functional form for the pure price index PLo, it is necessary to impose at least one additional test or axiom on PLo( p0, p1, q0, q1).

15.30 There is a potential problem with the use of the Edgeworth–Marshall price index (15.18) that has been noticed in the context of using the formula to make international comparisons of prices.
If the price levels of a very large country are compared to the price levels of a small country using formula (15.18), then the quantity vector of the large country may totally overwhelm the inﬂuence of the quantity vector corresponding to the small country.31 In technical terms, the Edgeworth– Marshall formula is not homogeneous of degree 0 in the components of both q0 and q1.
To prevent this problem from occurring in the use of the pure price index PK ( p0, p1, q0, q1) deﬁned by equation (15.17), it is asked that PLo satisfy the following invariance to proportional changes in current quantities test:32 PLo( p0, p1, q0, lq1)=PLo( p0, p1, q0, q1) for all p0, p1, q0, q1and all l > 0 (15:20) The two tests, the time reversal test (15.13) and the invariance test (15.20), make it possible to determine the precise functional form for the pure price index PLo deﬁned by formula (15.17): the pure price index PK must be the Walsh index PW deﬁned by formula (15.19).33 15.31 In order to be of practical use by statistical agencies, an index number formula must be able to be expressed as a function of the base period expenditure shares, s0 i , and the n price ratios, p1 i .
The Walsh price index deﬁned by the formula (15.19) can be rewritten in the following format: i , the current period expenditure shares, s1 i =p0 by equation (15.12).

The Walsh 30 Walsh (1921a, p. 103) endorsed PW as being the best index number formula: ‘‘We have seen reason to believe formula 6 better than for- mula 7.

Perhaps formula 9 is the best of the rest, but between it and Nos.
6 and 8 it would be diﬃcult to decide with assurance.’’ His for- mula 6 is PW deﬁned by equation (15.19) and his 9 is the Fisher ideal index, deﬁned QW ( p0, p1, q0, q1), is deﬁned as PW (q0, q1, p0, p1); i.e., the role of prices and quantities in deﬁnition (15.19) is interchanged.

If the Walsh quantity index is used to deﬂate the value ratio, an implicit price index is obtained, which is Walsh’s formula 8.
31 This is not likely to be a severe problem in the time series context, however, where the change in quantity vectors going from one period to the next is small.

32 This is the terminology used by Diewert (1992a, p. 216); Vogt (1980) was the ﬁrst to propose this test.

33 See section 7 in Diewert (2001).
quantity BASIC INDEX NUMBER THEORY p ﬃﬃﬃﬃﬃﬃﬃﬃﬃ q0 i q1 i p1 i q ﬃﬃﬃﬃﬃﬃﬃﬃﬃ q0 j q1 j p0 j p ﬃﬃﬃﬃﬃﬃﬃﬃ ﬃﬃﬃﬃﬃﬃﬃﬃﬃ s0 p0 i s1 i p1 i i p p1 i = p0 j = q ﬃﬃﬃﬃﬃﬃﬃﬃ s0 j s1 j ﬃﬃﬃﬃﬃﬃﬃﬃﬃ p0 j p1 j q p ﬃﬃﬃﬃﬃﬃﬃﬃ s0 i s1 i p ﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃ i =p0 p1 i q ﬃﬃﬃﬃﬃﬃﬃﬃ s0 j s1 j q ﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃ p0 j =p1 j (15:21) n P i=1 n P j=1 n P i=1 n P j=1 n P i=1 n P j=1 = = 15.32 The approach taken to index number theory in this section was to consider averages of various ﬁxed basket type price indices.
The ﬁrst approach was to take an even-handed average of the two primary ﬁxed basket indices: the Laspeyres and Paasche price indices.

These two primary indices are based on pricing out the baskets that pertain to the two periods (or locations) under consideration.

Taking an average of them led to the Fisher ideal price index PF deﬁned by equation (15.12).
The second approach was to average the basket quantity weights and then price out this average basket at the prices pertaining to the two situations under con- sideration.

This approach led to the Walsh price index, PW, deﬁned by equation (15.19).

Both of these indices can be written as a function of the base period expen- diture shares, s0 i , the current period expenditure shares, i , and the n price ratios, p1 s1 i .
Assuming that the statistical agency has information on these three sets of variables, which index should be used?

Experience with normal time series data has shown that these two indices will not diﬀer substantially and thus it is a matter of indiﬀerence which of these indices is used in practice.34 Both of these indices are examples of superlative indices, which are deﬁned in Chapter 17.

Note, however, that both of these indices treat the data pertaining to the two situations in a symmetric manner.
Hill35 commented on superlative price indices and the importance of a sym- metric treatment of the data as follows: Thus economic theory suggests that, i =p0 in general, a symmetric index that assigns equal weight to the two situations being compared is to be preferred to either the Laspeyres or Paasche indices on their own.
The precise choice of superlative index – whether Fisher, To¨ rnqvist or other superlative index – may be of only secondary importance as all the symmetric indices are likely to approximate each other, and the underlying theoretic index fairly closely, at least when the index number spread between the Laspeyres and Paasche is not very great (Hill (1993, p. 384)).

34 Diewert (1978, pp.


887–889) showed that these two indices will approximate each other to the second order around an equal price and quantity point.

Thus for normal time series data where prices and quantities do not change much going from the base period to the current period, the indices will approximate each other quite closely.

35 See also Hill (1988).


269 CONSUMER PRICE INDEX MANUAL: THEORY AND PRACTICE Annual weights and monthly price indices The Lowe index with monthly prices and annual base year quantities 15.33 It is now necessary to discuss a major prac- tical problem with the above theory of basket type it has been assumed that the indices.

Up to now, quantity vector q:(q1, q2, .

.

.


, qn) that appeared in the deﬁnition of the Lowe index, PLo( p0, p1, q) deﬁned by equation (15.15), is either the base period quantity vector q0 or the current period quantity vector q1 or an average of these two quantity vectors.

In fact, in terms of actual statistical agency practice, the quantity vector q is usually taken to be an annual quantity vector that refers to a base year, say b, that is prior to the base period for the prices, period 0.
Typically, a statistical agency will produce a consumer price index at a monthly or quarterly frequency, but for the sake of argument a monthly frequency will be assumed in what follows.
Thus a typical price index will have the form PLo( p0, pt, qb), where p0 is the price vector pertaining to the base period month for prices, month 0, pt is the price vector pertaining to the current period month for prices, say month t, and qb is a reference basket quantity vector that refers to the base year b, which is equal to or prior to month 0.36 Note that this Lowe index PLo( p0, pt, qb) is not a true Laspeyres index (because the annual quantity vector qb is not equal to the monthly quantity vector q0 in general).37 15.34 The question is: why do statistical agencies not pick the reference quantity vector q in the Lowe formula to be the monthly quantity vector q0 that pertains to transactions in month 0 (so that the index would reduce to an ordinary Laspeyres price index)?
There are two main reasons why this is not done: Most economies are subject to seasonal ﬂuctuations, and so picking the quantity vector of month 0 as the reference quantity vector for all months of the year would not be representative of transactions made throughout the year.

 Monthly household quantity or expenditure weights are usually collected by the statistical agency using a household expenditure survey with a relatively small sample.
Hence the resulting weights are usually sub- ject to very large sampling errors and so standard practice is to average these monthly expenditure or quantity weights over an entire year (or in some cases, over several years), in an attempt to reduce these sampling errors.

36 Month 0 is called the price reference period and year b is called the weight reference period.
37 Triplett (1981, p. 12) deﬁned the Lowe index, calling it a Laspeyres index, and calling the index that has the weight reference period equal to the price reference period a pure Laspeyres index.

Balk (1980c, p. 69), however, asserted that although the Lowe index is of the ﬁxed base type; it is not a Laspeyres price index.

Triplett also noted the hybrid share representation for the Lowe index deﬁned by equations (15.15) and (15.16).
Triplett noted that the ratio of two Lowe indices using the same quantity weights was also a Lowe index.

Baldwin (1990, p. 255) called the Lowe index an annual basket index.

270 The index number problems that are caused by seasonal monthly weights are studied in more detail in Chapter 22.
For now, it can be argued that the use of annual weights in a monthly index number formula is simply a method for dealing with the seasonality problem.38 15.35 One problem with using annual weights cor- responding to a perhaps distant year in the context of a monthly consumer price index must be noted at this point: if there are systematic (but divergent) trends in commodity prices and households increase their pur- chases of commodities that decline (relatively) in price and reduce their purchases of commodities that increase (relatively) in price, then the use of distant quantity weights will tend to lead to an upward bias in this Lowe index compared to one that used more current weights, as will be shown below.
This observation sug- gests that statistical agencies should strive to get up-to- date weights on an ongoing basis.

15.36 It is useful to explain how the annual quantity vector qb could be obtained from monthly expenditures on each commodity during the chosen base year b.

Let the month m expenditure of the reference population in the base year b for commodity i be vb, m and let the and qb, m corresponding price and quantity be pb, m respectively.
Of course, value, price and quantity for each commodity are related by the following equa- tions: i i i i =pb, m vb, m i qb, m i where i=1, .

.

.

, n and m=1, .

.

.

, 12 (15:22) For each commodity i, the annual total, qb i , can be obtained by price deﬂating monthly values and summing over months in the base year b as follows: 12 i = P qb m=1 vb, m i pb, m i 12 = P m=1 qb, m i ; i=1, .

.

.

, n (15:23) where equation (15.22) was used to derive the second equation in (15.23).
In practice, the above equations will be evaluated using aggregate expenditures over closely related commodities and the price pb, m i will be the month m price index for this elementary commodity group i in year b relative to the ﬁrst month of year b.

15.37 For some purposes, it is also useful to have annual prices by commodity to match up with the annual quantities deﬁned by equation (15.23).
Following national income accounting conventions, a reasonable39 38 In fact, the use of the Lowe index PLo( p0, pt, qb ) in the context of seasonal commodities corresponds to Bean and Stine’s (1924, p. 31) Type A index number formula.

Bean and Stine made three additional suggestions for price indices in the context of seasonal commodities.

Their contributions are evaluated in Chapter 22.

39 These annual commodity prices are essentially unit value prices.
Under conditions of high inﬂation, the annual prices deﬁned by equation (15.24) may no longer be ‘‘reasonable’’ or representative of prices during the entire base year because the expenditures in the ﬁnal months of the high inﬂation year will be somewhat artiﬁcially blown up by general inﬂation.

Under these conditions, the annual prices and annual commodity expenditure shares should be interpreted with caution.
For more on dealing with situations where there is high inﬂation within a year, see Hill (1996).

price pb i to match up with the annual quantity qb is the i value of total consumption of commodity i in year b divided by qb i .

Thus we have: 12 i P pb m=1 vb, m i =qb i i=1, .

.

.
, n using (15:23) 12 P m=1 vb, m i vb, m i =pb, m i = 12 P m=1 12 = P m=1 1 sb, m i ( pb, m i )1 (15:24) PLo( p0, pt, qb) BASIC INDEX NUMBER THEORY denominator can be calculated if the base year com- modity expenditure shares, sb i , are known along with the price ratios that compare the prices of commodity i in month 0, p0 i , with the corresponding annual average prices in the base year b, pb i : 15.39 There is another convenient formula for eval- uating the Lowe index, PLo( p0, pt,qb), and that is to use the hybrid weights formula (15.15).
In the present con- text, the formula becomes: n P i=1 = n P i=1 n P i=1 iqb pt i i qb p0 i ( pt i=p0 i )p0 i qb i n P i=1 i qb p0 i n =P i=1 pt i p0 i s0b i (15:27) where the share of annual expenditure on commodity i in month m of the base year is where the hybrid weights s0b and the quantities of year b are deﬁned by i using the prices of month 0 sb, m i vb, m i 12 P k=1 vb;k i ; i=1, .

.

.

, n (15:25) i , pb, 2 i , .

.

.


, pb, 12 Thus the annual base year price for commodity i, pb i turns out to be a monthly expenditure weighted har- monic mean of the monthly prices for commodity i in the base year, pb, 1 .

15.38 Using the annual commodity prices for the base year deﬁned by equation (15.24), a vector of these prices can be deﬁned as pb [ pb n].
Using this deﬁnition, the Lowe index PLo( p0, pt, qb) can be ex- pressed as a ratio of two Laspeyres indices, where the price vector pb plays the role of base period prices in each of the two Laspeyres indices: 1, .

.

.
.

.

, pb i PLo( p0, pt, qb) = iqb pt i i qb p0 i = iqb pt i i qb p0 i n P i=1 n P i=1 n P i=1 n P i=1 i qb pb i i qb pb i i ( pt sb i=pb i ) sb i ( p0 i =pb i ) n P i=1 n P i=1 n P i=1 n P i=1 =PL( pb, pt, qb)=PL( pb, p0, qb) (15:26) where the Laspeyres formula PL was deﬁned by equa- tion (15.5).
Thus the above equation shows that the Lowe monthly price index comparing the prices of month 0 to those of month t using the quantities of base year b as weights, PLo( p0, pt, qb), is equal to the Las- peyres index that compares the prices of month t to those of year b, PL( pb, pt, qb), divided by the Laspeyres index that compares the prices of month 0 to those of year b, PL( pb, p0, qb).
Note that the Laspeyres index in the numerator can be calculated if the base year com- modity expenditure shares, sb i , are known along with the price ratios that compare the prices of commodity i in month t, pt i, with the corresponding annual average prices in the base year b, pb i .

The Laspeyres index in the i=1, .

.

.


, n s0b i = ; i qb p0 i n P j qb pb j j=1 i qb pb i ( p0 h pb j qb j ( p0 n P j=1 i =pb i ) j =pb j ) i (15:28) The second equation in (15.28) shows how the base year i qb expenditures, pb i , can be multiplied by the commodity i =pb price indices, p0 i , in order to calculate the hybrid shares.

i=pb 15.40 There is one additional formula for the Lowe index, PLo( p0, pt, qb), that will be exhibited.
Note that the Laspeyres decomposition of the Lowe index deﬁned by the third term in equation (15.26) involves the long-term price relatives, pt i , which compare the prices in month t, pt i, with the possibly distant base year prices, pb i , and that the hybrid share decomposition of the Lowe index deﬁned by the third term in equation (15.27) involves the long-term monthly price relatives, pt i , which compare the prices in month t, pt i, with the base month prices, p0 i .
Both of these formulae are unsatisfactory in practice because of sample attrition: each month, a substantial fraction of commodities disappears from the market- place.

Thus it is useful to have a formula for updating the previous month’s price index using just month-over- month price relatives.

In other words, long-term price relatives disappear at too fast a rate to make it viable, in practice, to base an index number formula on their use.
The Lowe index for month t+1, PLo( p0, pt+1, qb), can be the Lowe index for month t, written in terms of PLo( p0, pt, qb), and an updating factor as follows: i=p0 PLo( p0, pt+1, qb) n P i=1 n P i=1 pt+1 i qb i = i qb p0 i 2 6 6 4 n P i=1 n P i=1 2 =PLo( p0, pt+1, qb) 6 6 4 3 7 7 5 iqb pt i i qb p0 i 2 6 6 4 3 7 7 5 pt+1 i qb i iqb pt i n P i=1 n P i=1 3 n P i=1 n P i=1 pt+1 i qb i iqb pt i 7 7 5 271 CONSUMER PRICE INDEX MANUAL: THEORY AND PRACTICE 2 =PLo( p0, pt+1, qb) 6 6 6 6 4 =PLo( p0, pt+1, qb) P n i=1 3 7 7 7 7 5 n P i=1 pt+1 i pt i !
iqb pt i n P i=1 iqb pt i pt+1 i pt i stb i (15:29) where the hybrid weights stb i are deﬁned by: stb i iqb pt i n P jqb pt j j=1 ; i=1, .

.

.
.

.

, n (15:30) =pt Thus the required updating factor, going from month t to month t+1, is the chain link index Pn i), which uses the hybrid share weights stb i corresponding to month t and base year b. i ( pt+1 i i=1stb 15.41 The Lowe index PLo( p0, pt, qb) can be re- garded as an approximation to the ordinary Laspeyres index, PL( p0, pt, q0), that compares the prices of the base month 0, p0, to those of month t, pt, using the quantity vectors of month 0, q0, as weights.
It turns out that there is a relatively simple formula that relates these two indices.

In order to explain this formula, it is ﬁrst necessary to make a few deﬁnitions.

Deﬁne the ith price relative between month 0 and month as ri pt i=p0 i ; i=1, .

.

.
.

.

, n (15:31) The ordinary Laspeyres price index, going from month 0 to t, can be deﬁned in terms of these price relatives as follows: PL( p0, pt, q0) n P i=1 pt i p0 i i q0 p0 i i q0 p0 i n P i=1 n pt iq0 i = n P i=1 n P i=1 n =P i=1 i q0 p0 i pt i p0 i i =P s0 i ri r* s0 i=1 (15:32) where the month 0 expenditure shares s0 follows: i are deﬁned as s0 i i q0 p0 i n P j q0 p0 j j=1 ; i=1, .

.

.
.

.

, n (15:33) 15.42 Deﬁne the ith quantity relative ti as the ratio of the quantity of commodity i used in the base year b, qb i , to the quantity used in month 0, q0 i , as follows: ti qb i =q0 i ; i=1, .

.

.
, n (15:34) The Laspeyres quantity index, QL(q0, qb, p0), that compares quantities in year b, qb, to the corresponding quantities in month 0, q0, using the prices of month 0, p0, as weights can be deﬁned as a weighted average of 272 the quantity ratios ti as follows: QL(q0, qb, p0) i qb p0 i i q0 p0 i = n P i=1 qb i q0 i i q0 p0 i n P i=1 i q0 p0 i n =P i=1 qb i q0 i s0 i s0 i ti using definition (15:34) n P i=1 n P i=1 n =P i=1 t* (15:35) 15.43 Using formula (A15.2.4) in Appendix 15.2 to this chapter, the relationship between the Lowe index PLo( p0, pt, qb) that uses the quantities of year b as weights to compare the prices of month t to month 0, and the corresponding ordinary Laspeyres index PL( p0, pt, q0) that uses the quantities of month 0 as weights is the following one: n P i=1 n P i=1 PLo( p0, pt, qb) i qb p0 i iqb pt i =PL( p0, pt, q0)+ n P i=1 (rir*)(tit*)s0 i QL(q0, qb, p0) (15:36) Thus the Lowe price index using the quantities of year b as weights, PLo( p0, pt, qb), is equal to the usual Laspeyres index using the quantities of month 0 as weights, PL( p0, pt, q0), plus a covariance term Pn i=1(ri r*) (ti i between the price relatives ri pt t*)s0 i and the quantity relatives ti qb i , divided by the Laspeyres quantity index QL(q0, qb, p0) between month 0 and base year b. i =q0 i=p0 15.44 Formula (15.36) shows that the Lowe price index will coincide with the Laspeyres price index if the covariance
 or correlation between the month 0 to t price relatives ri pt i=p0 i and the month 0 to year b quantity relatives ti qb i =q0 is zero.
Note that this cov- i ariance will be zero under three diﬀerent sets of conditions: if the month t prices are proportional to the month 0 prices so that all ri=r*; if the base year b quantities are proportional to the month 0 quantities so that all ti=t*; if the distribution of the relative prices ri is indepen- dent of the distribution of the relative quantities ti.
The ﬁrst two conditions are unlikely to hold empirically, but the third is possible, at least approximately, if con- sumers do not systematically change their purchasing habits in response to changes in relative prices.

15.45 If this covariance in formula (15.36) is nega- tive, then the Lowe index will be less than the Laspeyres index.

Finally, if the covariance is positive, then the Lowe index will be greater than the Laspeyres index.
Although the sign and magnitude of the covariance term, Pn i=1(ri r*)(ti t*)s0 i , is ultimately an empirical it is possible to make some reasonable con- matter, jectures about its likely sign.
If the base year b precedes the price reference month 0 and there are long-term trends in prices, then it is likely that this covariance is positive and hence that the Lowe index will exceed the corresponding Laspeyres price index;40 i.e., where the price relatives ri pt i are deﬁned by equation (15.31) and their share-weighted average r* by equation (15.32) and the ui, u* and QL are deﬁned as follows: i=p0 PLo( p0, pt, qb) > PL( p0, pt, q0) (15:37) ui qt i=q0 i ; i=1, .

.

.
.

.

, n (15:40) BASIC INDEX NUMBER THEORY (15:38) PP( p0, pt, qt) < PL( p0, pt, q0) (15:42) i=p0 i =qt i i =q0 To see why the covariance is likely to be positive, sup- pose that there is a long-term upward trend in the price of commodity i so that ri r* ( pt i ) r* is positive.
With normal consumer substitution responses,41 qt i=q0 i less an average quantity change of this type is likely to be negative, or, upon taking reciprocals, q0 less an average quantity change of this (reciprocal) type is likely to be positive.

But if the long-term upward trend in prices has persisted back to the base year b, then ti t* (qb i ) t* is also likely to be positive.

Hence, the covariance will be positive under these circumstances.
Moreover, the more distant is the base year b from the base month 0, the bigger the residuals ti t* are likely to be and the bigger will be the positive covariance.

Simi- larly, the more distant is the current period month t from the base period month 0, the bigger the residuals ri r* are likely to be and the bigger will be the positive co- variance.
Thus, under the assumptions that there are long-term trends in prices and normal consumer sub- stitution responses, the Lowe index will normally be greater than the corresponding Laspeyres index.
15.46 Deﬁne the Paasche index between months 0 and t as follows: PP( p0, pt, qt) n P i=1 n P i=1 iqt pt i i qt p0 i As discussed in paragraphs 15.18 to 15.23, a reasonable target index to measure the price change going from month 0 to t is some sort of symmetric average of the Paasche index PP( p0, pt, qt), deﬁned by formula (15.38), and the corresponding Laspeyres index, PL( p0, pt, q0), deﬁned by formula (15.32).
Adapting equation (A15.1.5) in Appendix 15.1, the relationship between the Paasche and Laspeyres indices can be written as follows: PP( p0, pt, qt)=PL( p0, pt, q0)+ n P i=1 (ri r*)(ui u*)s0 i QL(q0, qt, p0) (15:39) 40 For this relationship to hold, it is also necessary to assume that households have normal substitution eﬀects in response to these long- term trends in prices; i.e., if a commodity increases (relatively) in price, its consumption will decline (relatively) and if a commodity decreases relatively in price, its consumption will increase relatively.
41 Walsh (1901, pp.

281–282) was well aware of consumer substitution eﬀects, as can be seen in the following comment which noted the basic problem with a ﬁxed basket index that uses the quantity weights of a single period: ‘‘The argument made by the arithmetic averagist sup- poses that we buy the same quantities of every class at both periods in spite of the variation in their prices, which we rarely, if ever, do.
As a rough proposition, we – a community – generally spend more on ar- ticles that have risen in price and get less of them, and spend less on articles that have fallen in price and get more of them.’’ n u* P i=1 i ui=QL(q0, qt, p0) s0 (15:41) and the month 0 expenditure shares s0 i are deﬁned by the identity (15.33).
Thus u* is equal to the Laspeyres quantity index between months 0 and t. This means that the Paasche price index that uses the quantities of month t as weights, PP( p0, pt, qt), is equal to the usual Laspeyres index using the quantities of month 0 as weights, PL( p0, pt, q0), plus a covariance term Pn i=1(ri r*) (ui i between the price relatives ri pt u*)s0 i=p0 i and the quan- tity relatives ui qt i=q0 i , divided by the Laspeyres quan- tity index QL(q0, qt, p0) between month 0 and month t. 15.47 Although the sign and magnitude of the co- variance term, Pn i=1(ri r*)(ui u*)s0 is again an i , empirical matter, it is possible to make a reasonable conjecture about its likely sign.
If there are long-term trends in prices and consumers respond normally to price changes in their purchases, then it is likely that this covariance is negative and hence the Paasche index will be less than the corresponding Laspeyres price index; i.e., i=q0 i=p0 To see why this covariance is likely to be negative, sup- pose that there is a long-term upward trend in the price of commodity i 42 so that ri r* ( pt i ) r* is posi- tive.
With normal consumer substitution responses, qt i=q0 i less an average quantity change of this type is likely to be negative.

Hence ui u* (qt i ) u* is likely to be negative.

Thus, the covariance will be negative under these circumstances.
Moreover, the more distant is the base month 0 from the current month t, the bigger in magnitude the residuals ui u* are likely to be and the bigger in magnitude will be the negative covariance.43 Similarly, the more distant is the current period month t from the base period month 0, the bigger the residuals ri r* will probably be and the bigger in magnitude will be the covariance.
Thus under the assumptions that there are long-term trends in prices and normal consumer substitution responses, the Laspeyres index will be greater than the corresponding Paasche index, with the divergence likely to grow as month t becomes more distant from month 0.

15.48 Putting the arguments in the three previous it can be seen that under the paragraphs together, 42 The reader can carry through the argument if there is a long-term relative decline in the price of the ith commodity.
The argument required to obtain a negative covariance requires that there be some diﬀerences in the long-term trends in prices; i.e., if all prices grow (or fall) at the same rate, there will be price proportionality and the co- variance will be zero.

43 However, QL=u* may also be growing in magnitude, so the net eﬀect on the divergence between PL and PP is ambiguous.
273 CONSUMER PRICE INDEX MANUAL: THEORY AND PRACTICE assumptions that there are long-term trends in prices and normal consumer substitution responses, the Lowe price index between months 0 and t will exceed the corresponding Laspeyres price index, which in turn will exceed the corresponding Paasche price index; i.e., under these hypotheses, PLo(p0, pt, qb) > PL( p0, pt, q0) > PP( p0, pt, qt) (15:43) Thus, if the long-run target price index is an average of the Laspeyres and Paasche indices, it can be seen that the Laspeyres index will have an upward bias relative to this target index and the Paasche index will have a downward bias.
In addition, if the base year b is prior to the price reference month, month 0, then the Lowe index will also have an upward bias relative to the Laspeyres index and hence also to the target index.

The Lowe index and mid-year indices 15.49 The discussion in the previous paragraph assumed that the base year b for quantities preceded the base month for prices, month 0.
If the current period month t is quite distant from the base month 0, however, then it is possible to think of the base year b as referring to a year that lies between months 0 and t. If the year b does fall between months 0 and t, then the Lowe index becomes a mid-year index.44 It turns out that the Lowe mid-year index no longer has the upward biases indicated by the inequalities in the inequality (15.43) under the assumption of long-term trends in prices and normal substitution responses by quantities.
15.50 It is now assumed that the base year quantity vector qb corresponds to a year that lies between months 0 and t. Under the assumption of long-term trends in prices and normal substitution eﬀects so that there are also long-term trends in quantities (in the opposite direction to the trends in prices so that if the ith com- modity price is trending up, then the corresponding ith quantity is trending down), it is likely that the inter- 44 The concept of the mid-year index can be traced to Hill (1998, p. 46): When inﬂation has to be measured over a speciﬁed sequence of years, such as a decade, a pragmatic solution to the problems raised above would be to take the middle year as the base year.
This can be justiﬁed on the grounds that the basket of goods and services purchased in the middle year is likely to be much more representative of the pattern of con- sumption over the decade as a whole than baskets purchased in either the ﬁrst or the last years.

Moreover, choosing a more representative basket will also tend to reduce, or even eliminate, any bias in the rate of inﬂation over the decade as a whole as compared with the increase in the CoL index.
Thus, in addition to introducing the concept of a mid-year index, Hill also introduced the terminology representativity bias.

Baldwin (1990, pp.

255–256) also introduced the term representativeness: ‘‘Here representativeness [in an index number formula] requires that the weights used in any comparison of price levels are related to the volume of purchases in the periods of comparison.’’ However, this basic idea dates back to Walsh (1901, p. 104; 1921a, p. 90).
Baldwin (1990, p. 255) also noted that his concept of representa- tiveness was the same as Drechsler’s (1973, p. 19) concept of char- acteristicity.

For additional material on mid-year indices, see Schultz (1999) and Okamoto (2001).

Note that the mid-year index concept could be viewed as a close competitor to Walsh’s (1901, p. 431) multi- year ﬁxed basket index where the quantity vector was chosen to be an arithmetic or geometric average of the quantity vectors in the span of periods under consideration.
274 mediate year quantity vector will lie between the monthly quantity vectors q0 and qt.

The mid-year Lowe index, PLo( p0, pt, qb), and the Laspeyres index going from month 0 to t, PL( p0, pt, q0), will still satisfy the exact relationship given by equation (15.36).
Thus PLo( p0, pt, qb) will equal PL( p0, pt, q0) plus the covariance term [Pn i ]=QL(q0, qb, p0), where QL(q0, qb, p0) is the Laspeyres quantity index going from month 0 to t. This covariance term is likely to be negative so that i=1(ri r*) (ti t*)s0 PL( p0, pt, q0) > PLo( p0, pt, qb): (15:44) i, qb i =q0 i i and qt To see why this covariance is likely to be negative, suppose that there is a long-term upward trend in the price of commodity i so that ri r* ( pit=p0 i ) r* is positive.
With normal consumer substitution responses, qi will tend to decrease relatively over time and since qb i is assumed to be between q0 less an aver- age quantity change of this type is likely to be negative.

Hence ui u* (qb i ) t* is likely to be negative.

Thus, the covariance is likely to be negative under these circumstances.
Therefore, under the assump- tions that the quantity base year falls between months 0 and t and that there are long-term trends in prices and normal consumer substitution responses, the Las- peyres index will normally be larger than the corre- sponding Lowe mid-year index, with the divergence probably growing as month t becomes more distant from month 0. i =q0 15.51 It can also be seen that under the above assumptions, the mid-year Lowe index is likely to be greater than the Paasche index between months 0 and t; i.e., PLo( p0, pt, qb) > PP( p0, pt, qt) (15:45) To see why the above inequality is likely to hold, think of qb starting at the month 0 quantity vector q0 and then trending smoothly to the month t quantity vector qt.
When qb=q0, the Lowe index PLo( p0, pt, qb) becomes the Laspeyres index PL( p0, pt, q0).

When qb=qt, the Lowe index PLo( p0, pt, qb) becomes the Paasche index PP( p0, pt, qt).

Under the assumption of trending prices and normal substitution responses to these trending prices, it was shown earlier that the Paasche index will be less than the corresponding Laspeyres price index; that PP( p0, pt,qt) was less than PL( p0, pt,q0), recalling the inequality (15.42).
Thus, under the assumption of smoothly trending prices and quantities between months 0 and t, and assuming that qb is between q0 and qt, we will have i.e., PP( p0, pt, qt) < PLo( p0, pt, qb) < PL( p0, pt, q0) (15:46) Thus if the base year for the Lowe index is chosen to be in between the base month for the prices, month 0, and the current month for prices, month t, and there are trends in prices with corresponding trends in quantities that correspond to normal consumer substitution eﬀects, then the resulting Lowe index is likely to lie between the Paasche and Laspeyres indices going from months 0 to t. If the trends in prices and quantities are smooth, then choosing the base year half-way between periods 0 and t should give a Lowe index that is approxi- mately half-way between the Paasche and Laspeyres indices; hence it will be very close to an ideal target index between months 0 and t. This basic idea has been implemented by Okamoto (2001), using Japanese con- sumer data, and he found that the resulting mid-year indices approximated very closely to the corresponding Fisher ideal indices.
15.52 It should be noted that these mid-year indices can only be computed on a retrospective basis; i.e., they cannot be calculated in a timely fashion, as can Lowe indices that use a base year that is prior to month 0.

Thus mid-year indices cannot be used to replace the more timely Lowe indices.
The above material indicates, however, that these timely Lowe indices are likely to have an upward bias that is even bigger than the usual Laspeyres upward bias compared to an ideal target index, which was taken to be an average of the Paasche and Laspeyres indices.

15.53 All the inequalities derived in this section rest on the assumption of long-term trends in prices (and corresponding economic responses in quantities).
If there are no systematic long-run trends in prices, but only random ﬂuctuations around a common trend in all prices, then the above inequalities are not valid and the Lowe index using a prior base year will probably provide a perfectly adequate approximation to both the Paasche and Laspeyres indices.

There are, however, reasons for believing that there are some long-run trends in prices.
In particular: The computer chip revolution of the past 40 years has led to strong downward trends in the prices of products that use these chips intensively.

As new uses for chips have been developed over the years, the share of products that are chip intensive has grown and this implies that what used to be a relatively minor problem has become a more major problem.

 Other major scientiﬁc advances have had similar eﬀects.
For example, the invention of ﬁbre optic cable (and lasers) has led to a downward trend in tele- communications prices as obsolete technologies based on copper wire are gradually replaced.

 Since the end of the Second World War, a series of international trade agreements has dramatically reduced tariﬀs around the world.
These reductions, combined with improvements in transport technolo- gies, have led to a very rapid growth of international trade and remarkable improvements in international specialization.

Manufacturing activities in the more developed economies have gradually been outsourced to lower-wage to deﬂation in goods prices in most countries around the world.
In contrast, many services cannot be readily outsourced45 and so, on average, the price of services trends upwards while the price of goods trends downwards.

countries, leading BASIC INDEX NUMBER THEORY At the microeconomic level, there are tremendous diﬀerences in growth rates of ﬁrms.

Successful ﬁrms expand their scale, lower their costs, and cause less successful competitors to wither away with their higher prices and lower volumes.
This leads to a sys- tematic negative correlation between changes in item prices and the corresponding changes in item volumes that can be very large indeed.

Thus there is some a priori basis for assuming long-run divergent trends in prices.

Hence there is some basis for concern that a Lowe index that uses a base year for quantity weights that is prior to the base month for prices may be upwardly biased, compared to a more ideal target index.
The Young index 15.54 Recall the deﬁnitions for the base year quan- tities, qb i , given by equations (15.23) and (15.24) above.

The base year expenditure shares can be deﬁned in the usual way as follows: i , and the base year prices, pb sb i i qb pb i n P kqb pb k k=1 ; i=1, .

.

.

, n (15:47) 1, .

.

.

, sb Deﬁne the vector of base year expenditure shares in the usual way as sb [sb n].
These base year expenditure shares were used to provide an alternative formula for the base year b Lowe price index going from month 0 to t, deﬁned in equation (15.26) as PLo .
Rather = Pn ( p0, pt, qb)= Pn i=pb i than using this index as their short-run target index, many statistical agencies use the following closely rela- ted index: i=1sb i=1sb i p0 i pt i =pb i PY ( p0, pt, sb) P n i=1 i pt sb i=p0 i (15:48) This type of index was ﬁrst deﬁned by the English economist, Arthur Young (1812).46 Note that there is a change in focus when the Young index is used compared to the other indices proposed earlier in this chapter.
Up to this point, the indices proposed have been of the ﬁxed basket type (or averages of such indices) where a com- modity basket that is somehow representative for the two periods being compared is chosen and then ‘‘pur- chased’’ at the prices of the two periods and the index is taken to be the ratio of these two costs.
In contrast, for the Young index, representative expenditure shares are chosen that pertain to the two periods under consideration, and then these shares are used to calcu- late the overall index as a share-weighted average of the individual price ratios, pt i=p0 i .
Note that this view of index number theory, based on the share-weighted average of price ratios, is a little diﬀerent from the view taken at the beginning of this chapter, which saw the 45 Some services, however, can be internationally outsourced; e.g., call centres, computer programming and airline maintenance.

46 This formula is attributed to Young by Walsh (1901, p. 536; 1932, p. 657).
275 CONSUMER PRICE INDEX MANUAL: THEORY AND PRACTICE index number problem as that of decomposing a value ratio into the product of two terms, one of which expresses the amount of price change between the two periods and the other which expresses the amount of quantity change.47 sometimes 15.55 Statistical agencies regard the Young index, deﬁned above, as an approximation to the Laspeyres price index PL( p0, pt, q0).

Hence, it is of interest to see how the two indices compare.
Deﬁning the long-term monthly price relatives going from month 0 to t as ri pt i and using deﬁnitions (15.32) and (15.48): i=p0 PY ( p0, pt, sb)PL( p0, pt, q0) P n i=1 sb i s0 i pt i p0 i pt i p0 i n P i=1 pt i p0 i sb i s0 i sb i s0 i ri sb i s0 i rir* n =P i=1 n =P i=1 n =P i=1 n +r* P i=1 sb i s0 i n =P i=1 sb i s0 i rir* (15:49) i=1s0 i=1sb i =Pn i =1 and deﬁning r* Pn since Pn i=1 i ri=PL( p0, pt, q0).
Thus the Young index PY( p0, pt, sb) s0 is equal to the Laspeyres index PL( p0, pt, q0), plus the covariance between the diﬀerence in the annual shares pertaining to year b and the month 0 shares, sb i , and the deviations of the relative prices from their mean, ri r*.

i s0 15.56 It is no longer possible to guess at what the likely sign of the covariance term is.
The question is no longer whether the quantity demanded goes down as the price of commodity i goes up (the answer to this question is usually ‘‘yes’’) but the new question is: does the share 47 Fisher’s 1922 book is famous for developing the value ratio decomposition approach to index number theory, but his introductory chapters took the share-weighted average point of view: ‘‘An index number of prices, then shows the average percentage change of prices from one point of time to another’’ (Fisher (1922, p. 3)).
Fisher went on to note the importance of economic weighting: ‘‘The preceding calculation treats all the commodities as equally important; conse- quently, the average was called ‘simple’.

If one commodity is more important than another, we may treat the more important as though it were two or three commodities, thus giving it two or three times as much ‘weight’ as the other commodity’’ (Fisher (1922, p. 6)).

Walsh (1901, pp.
Walsh (1901, pp.

430–431) considered both approaches: ‘‘We can either (1) draw some average of the total money values of the classes during an epoch of years, and with weighting so determined employ the geo- metric average of the price variations [ratios]; or (2) draw some average of the mass quantities of the classes during the epoch, and apply to them Scrope’s method.’’ Scrope’s method is the same as using the Lowe index.

Walsh (1901, pp.
Walsh (1901, pp.

88–90) consistently stressed the impor- tance of weighting price ratios by their economic importance (rather than using equally weighted averages of price relatives).

Both the value ratio decomposition approach and the share-weighted average approach to index number theory are studied from the axiomatic perspective in Chapter 16.

276 of expenditure go down as the price of commodity i goes up?

The answer to this question depends on the elasticity of demand for the product.
Let us provisionally assume, however, that there are long-run trends in commodity prices and if the trend in prices for commodity i is above the mean, then the expenditure share for the commodity trends down (and vice versa).

Thus we are assuming high elasticities or very strong substitution eﬀects.
Assuming also that the base year b is prior to month 0, then under these conditions, suppose that there is a long-term upward trend in the price of commodity i so that ri r* ( pt i ) r* is positive.

With the assumed very elastic consumer substitution responses, si will tend to decrease relatively over time and since sb is assumed to i i or sb i s0 be prior to s0 i will probably be positive.

Thus, the covariance is likely to be positive under these circumstances.
Hence with long-run trends in prices and very elastic responses of consumers to price changes, the Young index is likely to be greater than the corresponding Laspeyres index.

i is expected to be less than sb i=p0 i ; s0 i or sb commodity i 15.57 Assume that there are long-run trends in commodity prices.

If the trend in prices for commodity i is above the mean, then suppose that the expenditure share for the commodity trends up (and vice versa).
Thus we are assuming low elasticities or very weak substitution eﬀects.

Assume also that the base year b is prior to month 0 and suppose that there is a long-term upward trend in the price of so that ri r* ( pt i ) r* is positive.

With the assumed i=p0 very inelastic consumer substitution responses, si will tend to increase relatively over time and since sb is i assumed to be prior to s0 i , it will be the case that s0 is i greater than sb i s0 is negative.
Thus, the covar- i iance is likely to be negative under these circumstances.

Hence with long-run trends in prices and very inelastic responses of consumers to price changes, the Young index is likely to be less than the corresponding Laspeyres index.

15.58 The previous two paragraphs indicate that, a priori, it is not known what the likely diﬀerence between the Young index and the corresponding Laspeyres index will be.
If elasticities of substitution are close to one, then the two sets of expenditure shares, sb i , will be close to each other and the diﬀerence between the two indices will be close to zero.

If monthly expenditure shares have strong seasonal components, however, then the annual shares sb i could diﬀer substantially from the monthly shares s0 i .

i and s0 It is useful to have a formula for updating the previous month’s Young price index using just month- over-month price relatives.
The Young index for month t+1, PY( p0, pt+1, sb), can be written in terms of the Young index for month t, PY( p0, pt, sb), and an updating factor as follows: 15.59 PY ( p0, pt+1, sb) P n i=1 sb i pt+1 i p0 i =PY ( p0, pt, sb) n P i=1 n P i=1 i ( pt+1 sb i =p0 i ) i ( pt sb i=p0 i ) =p0 i ) Y ( p0, pt, sb) 1 P* n X i=1 sb i ( p0 i =pt i) BASIC INDEX NUMBER THEORY =PY ( p0, pt, sb) n P i=1 n P i=1 i qb pb i ( pt+1 i i qb pb i ( pt i=p0 i ) using definition (15:47) n P i=1 i qb pb i pt+1 i pt i pt i p0 i !
n P i=1 i qb pb i ( pt i=p0 i ) sb0t i ( pt+1 i =pt i) =PY ( p0, pt, sb) =PY ( p0, pt, sb) P n i=1 (15:50) where the hybrid weights sb0t i are deﬁned by sb0t i i ( pt i qb pb kqb pb i=p0 i ) k( pt k=p0 k) n P k=1 = i ( pt sb k( pt sb i=p0 i ) k=p0 k) n P k=1 i=1, .

.

.
.

.

, n (15:51) i i=p0 Thus the hybrid weights sb0t can be obtained from the base year weights sb i by updating them; i.e., by multi- plying them by the price relatives (or indices at higher levels of aggregation), pt i .

Thus the required updating factor, going from month t to month t+1, is the chain link index, Pn =pt i=1sb0t i), which uses the hybrid i share weights sb0t deﬁned by equation (15.51).
i ( pt+1 i 15.60 Even if the Young index provides a close approximation to the corresponding Laspeyres index, it is diﬃcult to recommend the use of the Young index as a ﬁnal estimate of the change in prices going from period 0 to t, just as it was diﬃcult to recommend the use of the Laspeyres index as the ﬁnal estimate of inﬂation going from period 0 to t. Recall that the problem with the Laspeyres index was its lack of symmetry in the treat- ment of the two periods under consideration; i.e., using the justiﬁcation for the Laspeyres index as a good ﬁxed basket index, there was an identical justiﬁcation for the use of the Paasche index as an equally good ﬁxed basket index to compare periods 0 and t. The Young index suﬀers from a similar lack of symmetry with respect to the treatment of the base period.
The problem can be explained as follows.

The Young index, PY( p0, pt, sb), deﬁned by equation (15.48) calculates the price change between months 0 and t treating month 0 as the base.

But there is no particular reason to necessarily treat month 0 as the base month other than convention.

Hence, if we treat month t as the base and use the same formula to measure the price change from month t back to month 0, the index PY ( pt, p0, sb)=Pn i =pb i ), would be appropriate.
This estimate of price change can then be made comparable to the original Young index by taking its reciprocal, leading to the following rebased Young index,48 P* Y ( pt, p0, sb), deﬁned as i=1sb i ( p0 48 Using Fisher’s [PY ( pt, p0, sb)] PY( p0, pt, sb).
(1922, p. 118) * ( p0, pt, sb) 1= is the time antithesis of the original Young index, terminology, PY = " n X i=1 i ( pt sb i=p0 i )1 #1 (15:52) Y ( p0, pt, sb), which uses the The rebased Young index, P* current month as the initial base period, is a share- weighted harmonic mean of the price relatives going from month 0 to month t, whereas the original Young index, PY( p0, pt, sb), is a share-weighted arithmetic mean of the same price relatives.
15.61 Fisher argued as follows that an index number formula should give the same answer no matter which period was chosen as the base: Either one of the two times may be taken as the ‘‘base’’.

Will it make a diﬀerence which is chosen?

Cer- tainly, it ought not and our Test 1 demands that it shall not.
More fully expressed, the test is that the formula for calculating an index number should be such that it will give the same ratio between one point of comparison and the other point, no matter which of the two is taken as the base (Fisher (1922, p. 64)).
15.62 The problem with the Young index is that not only does it not coincide with its rebased counterpart, but there is a deﬁnite inequality between the two indices, namely: Y ( p0, pt, sb) PY ( p0, pt, sb) P* (15:53) with a strict inequality provided that the period t price vector pt is not proportional to the period 0 price vector p0.49 A statistical agency that uses the direct Young index PY ( p0, pt, sb) will generally show a higher inﬂation rate than a statistical agency that uses the same raw data but uses the rebased Young index, P* Y ( p0, ptI , sb).
15.63 The inequality (15.53) does not tell us by how much the Young index will exceed its rebased time antithesis.
In Appendix 15.3, however, it is shown that to the accuracy of a certain second-order Taylor series approximation, holds following between the direct Young index and its time antithesis: relationship the PY ( p0, pt, sb) P* Y ( p0, pt, sb)+PY ( p0, pt, sb)Var e (15:54) 49 These inequalities follow from the fact that a harmonic mean of M positive numbers is always equal to or less than the corresponding arithmetic mean; see Walsh (1901, p. 517) or Fisher (1922, pp.

383–384).
383–384).

This inequality is a special case of Schlo¨ milch’s (1858) inequality; see Hardy, Littlewood and Po´ lya (1934, p. 26).

Walsh (1901, pp.

330–332) explicitly noted the inequality (15.53) and also noted that the corre- sponding geometric average would fall between the harmonic and arithmetic averages.
Walsh (1901, p. 432) computed some numerical examples of the Young index and found big diﬀerences between it and his ‘‘best’’ indices, even using weights that were representative for the periods being compared.

Recall that the Lowe index becomes the Walsh index when geometric mean quantity weights are chosen and so the Lowe index can perform well when representative weights are used.

This is not necessarily the case for the Young index, even using repre- sentative weights.
Walsh (1901, p. 433) summed up his numerical ‘‘In fact, Young’s experiments with the Young index as follows: method, in every form, has been found to be bad.’’ 277 CONSUMER PRICE INDEX MANUAL: THEORY AND PRACTICE where Var e is deﬁned as n Var e P i=1 sb i ei e* 2 (15:55) The deviations ei are deﬁned by 1+ei=ri/r* for i=1, .

.

.

, n where the ri and their weighted mean r* are deﬁned by ri pt i=p0 i ; i=1, .

.

.
.

.

, n; n r* P i=1 sb i ri (15:56) (15:57) to equal which turns out the direct Young index, PY( p0, pt, sb).

The weighted mean of the ei is deﬁned as n e* P i=1 (15:58) sb i ei which turns out to equal 0.
Hence the more dispersion there is in the price relatives pt i , to the accuracy of a second-order approximation, the more the direct Young index will exceed its counterpart that uses month t as the initial base period rather than month 0. i=p0 15.64 Given two a priori equally plausible index number formulae that give diﬀerent answers, such as the Young index and its time antithesis, Fisher (1922, p. 136) generally suggested taking the geometric aver- age of the two indices.50 A beneﬁt of this averaging is that the resulting formula will satisfy the time reversal test.
Thus rather than using either the base period 0 Young index, PY( p0, pt, sb), or the current period t Y ( p0, pt, sb), which is always below the Young index, P* base period 0 Young index if there is any dispersion in relative prices, it seems preferable to use the following index, which is the geometric average of the two alter- natively based Young indices:51 # Y ( p0, pt, sb) PY ( p0, pt, sb)P* P** Y ( p0, pt, sb) $1=2 (15:59) If the base year shares sb i happen to coincide with both i and st the month 0 and month t shares, s0 i respectively, it can be seen that the time-rectiﬁed Young index Y ( p0, pt, sb) deﬁned by equation (15.59) will coincide P** 50 We now come to a third use of these tests, namely, to ‘‘rectify’’ formulae, i.e., to derive from any given formula which does not .
.

.

.

This is satisfy a test another formula which does satisfy it; easily done by ‘‘crossing’’, that is, by averaging antitheses.

If a given formula fails to satisfy Test 1 [the time reversal test], its time antithesis will also fail to satisfy it; but the two will fail, as it were, in opposite ways, so that a cross between them (obtained by geome- trical averaging) will give the golden mean which does satisfy (Fisher (1922, p. 136)).
Actually the basic idea behind Fisher’s rectiﬁcation procedure was suggested by Walsh, who was a discussant for Fisher (1921), where Fisher gave a preview of his 1922 book: ‘‘We merely have to take any index number, ﬁnd its antithesis in the way prescribed by Professor Fisher, and then draw the geometric mean between the two’’ (Walsh (1921b, p. 542)).
51 This index is a base year weighted counterpart to an equally weighted index proposed by Carruthers, Sellwood and Ward (1980, p. 25) and Dale´ n (1992, p. 140) in the context of elementary index number formulae.

See Chapter 20 for further discussion of this unweighted index.

278 with the Fisher ideal price index between months 0 and t, PF( p0, pt, q0, qt) (which will also equal the Las- peyres and Paasche indices under these conditions).
Y deﬁned by equation Note also that (15.59) can be produced on a timely basis by a statistical agency.

the index P** The Divisia index and discrete approximations to it The Divisia price and quantity indices 15.65 The second broad approach to index number theory relies on the assumption that price and quantity data change in a more or less continuous way.
15.66 Suppose that the price and quantity data on the n commodities in the chosen domain of deﬁnition can be regarded as continuous functions of (continuous) time, say pi(t) and qi(t) for i=1, .

.

.

, n. The value of consumer expenditure at time t is V(t) deﬁned in the obvious way as: n V(t) P i=1 pi(t)qi(t) (15:60) 15.67 Now suppose that the functions pi (t) and qi (t) are diﬀerentiable.
Then both sides of the deﬁnition (15.60) can be diﬀerentiated with respect to time to obtain: n V 0(t)= P i=1 n i(t)qi(t)+P p0 i=1 pi(t)q0 i(t) (15:61) Divide both sides of equation (15.61) through by V(t) and using deﬁnition (15.60), the following equation is obtained: n P i=1 V 0(t) V(t) = i(t)qi(t)+P p0 pi(t)q0 i(t) n i=1 n P j=1 pj(t)qj(t) n =P i=1 p0 i(t) pi(t) n si(t)+P i=1 q0 i(t) qi(t) si(t) (15:62) where the time t expenditure share on commodity i, si (t), is deﬁned as: si(t) pi(t)qi(t) n P m=1 pm(t)qm(t) for i=1, 2, .
.

.

, n (15:63) 15.68 Divisia (1926, p. 39) argued as follows: sup- pose the aggregate value at time t, V(t), can be written as the product of a time t price level function, P(t) say, times a time t quantity level function, Q(t) say; i.e., we have: V(t)=P(t)Q(t) (15:64) Suppose further that the functions P(t) and Q(t) are diﬀerentiable.
Then diﬀerentiating the equation (15.64) yields: V 0(t)=P0(t)Q(t)+P(t)Q0(t) (15:65) Dividing both sides of equation (15.65) by V(t) and using equation (15.64) leads to the following equation: V 0(t) V(t) = P0(t) P(t) + Q0(t) Q(t) (15:66) economic approach to index number theory.

This con- nection is, however, best made after studying the eco- nomic approach to index number theory.

Since this material is rather technical, it has been relegated to Appendix 15.4.
BASIC INDEX NUMBER THEORY 15.69 Divisia compared the two expressions for the logarithmic value derivative, V0(t)/V(t), given by equa- tions (15.62) and (15.66), and he simply deﬁned the the aggregate price logarithmic rate of change of level, P0(t)/P(t), as the ﬁrst set of terms on the right-hand side of (15.62).

He also simply deﬁned the logarithmic rate of change of the aggregate quantity level, Q0(t)/Q(t), as the second set of terms on the right-hand side of equation (15.62).
That is, he made the following deﬁni- tions: P0(t) P(t) n P i=1 si(t) p0 i(t) pi(t) Q0(t) Q(t) n P i=1 si(t) q0 i(t) qi(t) (15:67) (15:68) 15.70 Deﬁnitions (15.67) and (15.68) are reasonable deﬁnitions for the proportional changes in the aggre- gate price and quantity (or quantity) levels, P(t) and Q(t).52 The problem with these deﬁnitions is that eco- nomic data are not collected in continuous time; they are collected in discrete time.
In other words, even though transactions can be thought of as occurring in con- tinuous time, no consumer records his or her purchases as they occur in continuous time; rather, purchases over a ﬁnite time period are cumulated and then recorded.

A similar situation occurs for producers or sellers of commodities; ﬁrms cumulate their sales over discrete periods of time for accounting or analytical purposes.
If it is attempted to approximate continuous time by shorter and shorter discrete time intervals, empirical price and quantity data can be expected to become increasingly erratic since consumers only make pur- chases at discrete points of time (and producers or sellers of commodities only make sales at discrete points to of approximate the continuous time price and quantity levels, P(t) and Q(t) deﬁned implicitly by equations (15.67) and (15.68), by discrete time approximations.
This can be done in two ways.

Either methods of numerical approximation can be used or assumptions can be made about the path taken through time by the functions pi (t) and qi (t) (i=1, .

.

.

, n).

The ﬁrst strategy is used in the following section.

For discussions of the second strategy, see Vogt (1977; 1978), Van Ijzeren (1987, pp.

8–12), Vogt and Barta (1997) and Balk (2000a).

is, however, still of some interest time).


It 15.71 There is a connection between the Divisia price and quantity levels, P(t) and Q(t), and the Discrete approximations to the continuous time Divisia index 15.72 In order to make operational the continuous time Divisia price and quantity levels, P(t) and Q(t) deﬁned by the diﬀerential equations (15.67) and (15.68), it is necessary to convert to discrete time.
Divisia (1926, p. 40) suggested a straightforward method for doing this conversion, which we now outline.

15.73 Deﬁne the following price and quantity (for- ward) diﬀerences: DP P(1) P(0) (15:69) Dpi pi(1) pi(0); i=1, .

.

.
.

.

, n (15:70) Using the above deﬁnitions: P(1) P(0) = P(0)+DP P(0) =1+ DP P(0) 1+ n P i=1 n P m=1 Dpiqi(0) pm(0)qm(0) using (15.67) when t=0 and approximating p0 diﬀerence Dpi i(0) by the = n P i=1 fpi(0)+Dpigqi(0) n P m=1 pm(0)qm(0) = =PL( p0, p1, q0, q1) n P i=1 n P m=1 pi(1)qi(0) pm(0)qm(0) (15:71) where pt:[ p1(t), .

.

.

, pn(t)] and qt:[q1(t), .

.

.

, qn(t)] for t=0,1.


Thus, it can be seen that Divisia’s discrete approximation to his continuous time price index is just the Laspeyres price index, PL, deﬁned above by equa- tion (15.5).

15.74 But now a problem noted by Frisch (1936, p. 8) occurs: instead of approximating the derivatives by the discrete (forward) diﬀerences deﬁned by equations (15.69) and (15.70), other approximations could be used and a wide variety of discrete time approximations could be obtained.
For example, instead of using for- ward diﬀerences and evaluating the index at time t=0, it would be possible to use backward diﬀerences and evaluate the index at time t=1.
These backward diﬀer- ences are deﬁned as: 52 If these deﬁnitions are applied (approximately) to the Young index studied in the previous section, then it can be seen that in order for the Young price index to be consistent with the Divisia price index, the base year shares should be chosen to be average shares that apply to the entire time period between months 0 and t. Dbpi pi(0)pi(1); i=1, .

.

.
.

.

, n (15:72) This use of backward diﬀerences leads to the following approximation for P(0)/P(1): 279 CONSUMER PRICE INDEX MANUAL: THEORY AND PRACTICE P(0) P(1) = P(1)+DbP P(1) =1+ DbP P(1) 1+ Dbpiqi(1) weights are somehow representative of the two periods under consideration.
n P i=1 n P m=1 pm(1)qm(1) using (15.67) when t=1 and approximating p0 diﬀerence Dbpi i(1) by the n P i=1 = fpi(1)+Dbpigqi(1) n P m=1 pm(1)qm(1) = n P i=1 n P m=1 pi(0)qi(1) pm(1)qm(1) = 1 PP( p0, p1, q0, q1) (15:73) where PP is the Paasche index deﬁned above by equation (15.6).
Taking reciprocals of both sides of equation (15.73) leads to the following discrete approximation to P(1)/P(0): P(1) P(0) PP (15:74) 15.75 Thus, as Frisch53 noted, both the Paasche and Laspeyres indices can be regarded as (equally valid) approximations to the continuous time Divisia price index.54 Since the Paasche and Laspeyres indices can diﬀer considerably in some empirical applica- tions, it can be seen that Divisia’s idea is not all that in determining a unique discrete time index helpful number formula.55 What is useful about the Divisia indices is the idea that as the discrete unit of time gets smaller, discrete approximations to the Divisia indices can approach meaningful economic indices under certain conditions.
Moreover, the Divisia concept is accepted as the ‘‘correct’’ one for index number theory, then the corresponding ‘‘correct’’ dis- crete time counterpart might be taken as a weighted average of the chain price relatives pertaining to the the adjacent periods under consideration, where if 53 ‘‘As the elementary formula of the chaining, we may get Laspeyres’ or Paasche’s or Edgeworth’s or nearly any other formula, according as we choose the approximation principle for the steps of the numerical integration’’ (Frisch (1936, p. 8)).
54 Diewert (1980, p. 444) also obtained the Paasche and Laspeyres approximations to the Divisia index, using a somewhat diﬀerent approximation argument.

He also showed how several other popular discrete time index number formulae could be regarded as approx- imations to the continuous time Divisia index.

55 Trivedi (1981) systematically examined the problems involved in ﬁnding a ‘‘best’’ discrete time approximation to the Divisia indices using the techniques of numerical analysis.
These numerical analysis techniques depend on the assumption that the ‘‘true’’ continuous time micro-price functions, pi (t), can be adequately represented by a poly- nomial approximation.

Thus we are led to the conclusion that the ‘‘best’’ discrete time approximation to the Divisia index depends on assumptions that are diﬃcult to verify.
280 Fixed base versus chain indices 15.76 In this section,56 we discuss the merits of using the chain system for constructing price indices in the time series context versus using the ﬁxed base system.57 15.77 The chain system58 measures the change in prices going from one period to a subsequent period using a bilateral index number formula involving the prices and quantities pertaining to the two adjacent periods.
These one-period rates of change (the links in the chain) are then cumulated to yield the relative levels of prices over the entire period under consideration.
Thus if the bilateral price index is P, the chain system generates the following pattern of price levels for the ﬁrst three periods: 1, P( p0, p1, q0, q1), P( p0, p1, q0, q1) P( p1, p2, q1, q2) (15:75) 15.78 In contrast, the ﬁxed base system of price levels, using the same bilateral index number formula P, simply computes the level of prices in period t relative to the base period 0 as P( p0, pt, q0, qt).
Thus the ﬁxed base pattern of price levels for periods 0,1 and 2 is: 1, P( p0, p1, q0, q1), P( p0, p2, q0, q2) (15:76) 15.79 Note that in both the chain system and the ﬁxed base system of price levels deﬁned by the formulae (15.75) and (15.76), the base period price level is set equal to 1.

The usual practice in statistical agencies is to set the base period price level equal to 100.

If this is done, then it is necessary to multiply each of the num- bers in the formulae (15.75) and (15.76) by 100.
15.80 Because of the diﬃculties involved in obtain- ing current period information on quantities (or equivalently, on expenditures), many statistical agencies loosely base their consumer price index on the use of the Laspeyres formula (15.5) and the ﬁxed base system.

Therefore, it is of interest to look at some of the possible problems associated with the use of ﬁxed base Laspeyres indices.

56 This section is largely based on the work of Hill (1988; 1993, pp.

385–390).
385–390).

57 The results in Appendix 15.4 provide some theoretical support for the use of chain indices in that it is shown that under certain condi- tions, the Divisia index will equal an economic index.

Hence any dis- crete approximation to the Divisia index will approach the economic index as the time period gets shorter.

Thus under certain conditions, chain indices will approach an underlying economic index.
58 The chain principle was introduced independently into the eco- nomics literature by Lehr (1885, pp.

45–46) and Marshall (1887, p. 373).

Both authors observed that the chain system would mitigate the diﬃculties arising from the introduction of new commodities into the economy, a point also mentioned by Hill (1993, p. 388).

Fisher (1911, p. 203) introduced the term ‘‘chain system’’.
15.81 The main problem with the use of ﬁxed base Laspeyres indices is that the period 0 ﬁxed basket of commodities that is being priced out in period t can often be quite diﬀerent from the period t basket.
Thus if there are systematic trends in at least some of the prices and quantities59 in the index basket, the ﬁxed base Laspeyres price index PL( p0, pt, q0, qt) can be quite dif- ferent from the corresponding ﬁxed base Paasche price index, PP( p0, pt, q0, qt).60 This means that both indices are likely to be an inadequate representation of the movement in average prices over the time period under consideration.
15.82 The ﬁxed base Laspeyres quantity index can- not be used for ever: eventually, the base period quan- tities q0 are so far removed from the current period quantities qt that the base must be changed.
Chaining is merely the limiting case where the base is changed each period.61 15.83 The main advantage of the chain system is that under normal conditions, chaining will reduce the spread between the Paasche and Laspeyres indices.62 These two indices each provide an asymmetric per- spective on the amount of price change that has occur- red between the two periods under consideration and it could be expected that a single point estimate of the aggregate price change should lie between these two estimates.
Thus the use of either a chained Paasche or Laspeyres index will usually lead to a smaller diﬀerence between the two and hence to estimates that are closer to the ‘‘truth’’.63 15.84 Hill (1993, p. 388), drawing on the earlier research of Szulc (1983) and Hill (1988, pp.

136–137), noted that it is not appropriate to use the chain system when prices oscillate or bounce.

This phenomenon can occur in the context of regular seasonal ﬂuctuations or in the context of price wars.
However, in the context of roughly monotonically changing prices and quantities, Hill (1993, p. 389) recommended the use of chained symmetrically weighted indices (see paragraphs 15.18 to 15.32).

The Fisher and Walsh indices are examples of symmetrically weighted indices.

15.85 It is possible to be a little more precise about the conditions under which to chain or not to chain.
Basically, chaining is advisable if the prices and quan- tities pertaining to adjacent periods are more similar 59 Examples of rapidly downward trending prices and upward trending quantities are computers, electronic equipment of all types, Internet access and telecommunication charges.

60 Note that PL( p0, pt, q0, qt) will equal PP( p0, pt, q0, qt) if either the two quantity vectors q0 and qt are proportional or the two price vectors p0 and pt are proportional.
Thus in order to obtain a diﬀerence between the Paasche and Laspeyres indices, non-proportionality in both prices and quantities is required.

61 Regular seasonal ﬂuctuations can cause monthly or quarterly data to ‘‘bounce’’—using the term coined by Szulc (1983, p. 548)—and chaining bouncing data can lead to a considerable amount of index ‘‘drift’’; i.e., if after 12 months, prices and quantities return to their levels of a year earlier, then a chained monthly index will usually not return to unity.
Hence, the use of chained indices for ‘‘noisy’’ monthly or quarterly data is not recommended without careful consideration.

62 See Diewert (1978, p. 895) and Hill (1988; 1993, pp.

387–388).

63 This observation will be illustrated with an artiﬁcial data set in Chapter 19.
BASIC INDEX NUMBER THEORY than the prices and quantities of more distant periods, since this strategy will lead to a narrowing of the spread between the Paasche and Laspeyres indices at each link.64 Of course, one needs a measure of how similar are the prices and quantities pertaining to two periods.

The similarity measures could be relative ones or abso- lute ones.

In the case of absolute comparisons, two vectors of the same dimension are similar if they are identical and dissimilar otherwise.
In the case of relative comparisons, two vectors are similar if they are pro- portional and dissimilar if they are non-proportional.65 Once a similarity measure has been deﬁned, the prices and quantities of each period can be compared to each other using this measure, and a ‘‘tree’’ or path that links all of the observations can be constructed where the most similar observations are compared with each other using a bilateral index number formula.66 Hill (1995) deﬁned the price structures between two countries to be more dissimilar the bigger the spread between PL and PP; i.e., the bigger is {PL/PP, PP/PL}.
The problem with this measure of dissimilarity in the price structures of the two countries is that it could be the case that PL=PP (so that the Hill measure would register a maximal degree of similarity), but p0 could be very diﬀerent from pt.

Thus there is a need for a more systematic study of similarity (or dissimilarity) measures in order to pick the ‘‘best’’ one that could be used as an input into Hill’s (1999a; 1999b; 2001) linking observations.
spanning tree algorithm for 64 Walsh, in discussing whether ﬁxed base or chained index numbers should be constructed, took for granted that the precision of all rea- sonable bilateral index number formulae would improve, provided that the two periods or situations being compared were more similar, and hence favoured the use of chained indices: ‘‘The question is really, in which of the two courses [ﬁxed base or chained index numbers] are we likely to gain greater exactness in the comparisons actually made?
Here the probability seems to incline in favor of the second course; for the conditions are likely to be less diverse between two contiguous periods than between two periods say ﬁfty years apart’’ (Walsh (1901, p. 206)).

Walsh (1921a, pp.

84–85) later reiterated his preference for chained index numbers.
Fisher also made use of the idea that the chain system would usually make bilateral comparisons between price and quantity data that were more similar, and hence the resulting comparisons would be more accurate: The index numbers for 1909 and 1910 (each calculated in terms of 1867– 1877) are compared with each other.

But direct comparison between 1909 and 1910 would give a diﬀerent and more valuable result.
To use a common base is like comparing the relative heights of two men by measuring the height of each above the ﬂoor, instead of putting them back to back and directly measuring the diﬀerence of level between the tops of their heads (Fisher (1911, p. 204)).

It seems, therefore, advisable to compare each year with the next, or, in other words, to make each year the base year for the next.

Such a procedure has been recommended by Marshall, Edgeworth and Flux.
It largely meets the diﬃculty of non-uniform changes in the Q’s, for any inequalities for successive years are relatively small (Fisher (1911, pp.

423–424)).

65 (Diewert (2002b) takes an axiomatic approach to deﬁning various indices of absolute and relative dissimilarity.)

66 Fisher (1922, pp.

271–276) hinted at the possibility of using spatial linking; i.e., of linking countries that are similar in structure.
The modern literature has, however, grown as a result of the pioneering eﬀorts of Robert Hill (1995; 1999a; 1999b; 2001).

Hill (1995) used the spread between the Paasche and Laspeyres price indices as an indicator of similarity, and showed that this criterion gives the same results as a criterion that looks at the spread between the Paasche and Laspeyres quantity indices.
281 CONSUMER PRICE INDEX MANUAL: THEORY AND PRACTICE 15.86 The method of linking observations explained in the previous paragraph, based on the similarity of the price and quantity structures of any two observations, may not be practical in a statistical agency context since the addition of a new period may lead to a reordering of the previous links.
The above ‘‘scientiﬁc’’ method for linking observations may be useful, however, in deciding whether chaining is preferable or whether ﬁxed base indices should be used while making month-to-month comparisons within a year.
15.87 Some index number theorists have objected to the chain principle on the grounds that it has no coun- terpart in the spatial context: They [chain indices] only apply to intertemporal comparisons, and in contrast to direct indices they are not applicable to cases in which no natural order or sequence exists.
Thus the idea of a chain index for example has no counterpart in interregional or interna- tional price comparisons, because countries cannot be sequenced in a ‘‘logical’’ or ‘‘natural’’ way (there is no k þ 1 nor k 1 country to be compared with country k) (von der Lippe (2001, p. 12)).67 This is of course correct, but the approach of Hill does lead to a ‘‘natural’’ set of spatial links.
Applying the same approach to the time series context will lead to a set of links between periods which may not be month-to- month but it will in many cases justify year-over-year linking of the data pertaining to the same month.

This problem is reconsidered in Chapter 22.

15.88 It is of some interest to determine if there are index number formulae that give the same answer when either the ﬁxed base or chain system is used.
Comparing the sequence of chain indices deﬁned by the expression (15.75) to the corresponding ﬁxed base indi- ces, it can be seen that we will obtain the same answer in all three periods if the index number formula P satisﬁes the following functional equation for all price and quantity vectors: P( p0, p2, q0, q2)=P( p0, p1, q0, q1)P( p1, p2, q1, q2) (15:77) If an index number formula P satisﬁes the equation (15.77), then P satisﬁes the circularity test.68 15.89 If it is assumed that the index number formula P satisﬁes certain properties or tests in addition to the circularity test above,69 then Funke, Hacker and Voeller (1979) showed that P must have the following 67 It should be noted that von der Lippe (2001, pp.
56–58) is a vigorous critic of all index number tests based on symmetry in the time series context, although he is willing to accept symmetry in the context of making international comparisons.

‘‘But there are good reasons not to insist on such criteria in the intertemporal case.

When no symmetry exists between 0 and t, there is no point in interchanging 0 and t’’ (von der Lippe (2001, p. 58)).
68 The test name is attributable to Fisher (1922, p. 413) and the concept originated from Westergaard (1890, pp.

218–219).
218–219).

69 The additional tests referred to above are: (i) positivity and con- tinuity of P( p0, p1, q0, q1) for all strictly positive price and quantity vectors p0, p1, q0, q1; (ii) the identity test; (iii) the commensurability test; (iv) P( p0, p1, q0, q1) is positively homogeneous of degree one in the components of p1, and (v) P( p0, p1, q0, q1) is positively homogeneous of degree zero in the components of q1.

282 functional form, originally established by Konu¨ s and Byushgens70 (1926, pp.
163–166):71 PKB( p0, p1, q0, q1) Q n i=1 ai p1 i p0 i (15:78) where the n constants ai satisfy the following restric- tions: n P i=1 ai=1 and ai > 0 for i=1, .

.

.

, n (15:79) Thus under very weak regularity conditions, the only price index satisfying the circularity test is a weighted geometric average of all the individual price ratios, the weights being constant through time.
15.90 An interesting special case of the family of indices deﬁned by equation (15.78) occurs when the weights ai are all equal.
In this case, PKB reduces to the Jevons (1865) index: PJ( p0, p1, q0, q1) Q n i=1 n 1 p1 i p0 i (15:80) i =p0 15.91 The problem with the indices deﬁned by Konu¨ s and Byushgens, and Jevons is that the individual price ratios, p1 i , have weights (either ai or 1/n) that are independent of the economic importance of com- modity i in the two periods under consideration.
Put another way, these price weights are independent of the quantities of commodity i consumed or the expenditures on commodity i during the two periods.

Hence, these indices are not really suitable for use by statistical agencies at higher levels of aggregation when expendi- ture share information is available.

15.92 The above results indicate that it is not useful to ask that the price index P satisfy the circularity test exactly.
It is nevertheless of some interest to ﬁnd index number formulae that satisfy the circularity test to some degree of approximation, since the use of such an index number formula will lead to measures of aggregate price change that are more or less the same no matter whether we use the chain or ﬁxed base systems.

Fisher (1922, p. 284) found that deviations from circularity using his data set and the Fisher ideal price index PF deﬁned by equation (15.12) above were quite small.
This relatively high degree of correspondence between ﬁxed base and 70 Konu¨ s and Byushgens show that the index deﬁned by equation (15.78) is exact for Cobb–Douglas (1928) preferences; see also Pollak (1983, pp.

119–120).

The concept of an exact index number formula is explained in Chapter 17.

71 The result in equation (15.78) can be derived using results in Eich- horn (1978, pp.

167–168) and Vogt and Barta (1997, p. 47).

A simple proof can be found in Balk (1995).


This result vindicates Irving Fisher’s (1922, p. 274) intuition that ‘‘the only formulae which con- form perfectly to the circular test are index numbers which have con- stant weights .

.

.

’’.

Fisher (1922, p. 275) went on to assert: ‘‘But, clearly, constant weighting is not theoretically correct.

If we compare 1913 with 1914, we need one set of weights; if we compare 1913 with 1915, we need, theoretically at least, another set of weights.

.

.

.
Similarly, turning from time to space, an index number for comparing the United States and England requires one set of weights, and an index number for comparing the United States and France requires, theoretically at least, another.’’ chain indices has been found to hold for other symme- trically weighted formulae, such as the Walsh index PW deﬁned by equation (15.19).72 In most time series applications of index number theory where the base year in ﬁxed base indices is changed every ﬁve years or so, it will not matter very much whether the statistical agency uses a ﬁxed base price index or a chain index, provided that a symmetrically weighted formula is used.73 The choice between a ﬁxed base price index or chain index will depend, of course, on the length of the time series considered and the degree of variation in the prices and quantities as we go from period to period.
The more prices and quantities are subject to large ﬂuctuations (rather than smooth trends), the less the correspon- dence.74 15.93 It is possible to give a theoretical explanation for the approximate satisfaction of the circularity test for symmetrically weighted index number formulae.
Another symmetrically weighted formula is the To¨ rnqvist index PT.75 The natural logarithm of this index is deﬁned as follows: ln PT ( p0, p1, q0, q1) P n i=1 1 2 i +s1 s0 i ln p1 i p0 i (15:81) if show that where the period t expenditure shares st i are deﬁned by equation (15.7).
Alterman, Diewert and Feenstra (1999, p. 61) the logarithmic price ratios ln ( pt i=pt1 ) trend linearly with time t and the expendi- i ture shares st i also trend linearly with time, then the index PT will satisfy the circularity test To¨ rnqvist exactly.76 Since many economic time series on prices and quantities satisfy these assumptions approximately, the index PT will satisfy the circularity test To¨ rnqvist approximately.
As is seen in Chapter 19, the To¨ rnqvist index generally closely approximates the symmetrically weighted Fisher and Walsh indices, so that for many economic time series (with smooth trends), all three of these symmetrically weighted indices will satisfy the circularity test to a high enough degree of approxima- 72 See, for example, Diewert (1978, p. 894)).

Walsh (1901, pp.
Walsh (1901, pp.

424 and 429) found that his three preferred formulae all approximated each other very well, as did the Fisher ideal for his artiﬁcial data set.

73 More speciﬁcally, most superlative indices (which are symmetrically weighted) will satisfy the circularity test to a high degree of approx- imation in the time series context.

See Chapter 17 for the deﬁnition of a superlative index.
It is worth stressing that ﬁxed base Paasche and Laspeyres indices are very likely to diverge considerably over a ﬁve- year period if computers (or any other commodity which has price and quantity trends that are quite diﬀerent from the trends in the other commodities) are included in the value aggregate under consideration (see Chapter 19 for some ‘‘empirical’’ evidence on this topic).

74 Again, see Szulc (1983) and Hill (1988).


75 This formula was implicitly introduced in To¨ rnqvist (1936) and explicitly deﬁned in To¨ rnqvist and To¨ rnqvist (1937).

76 This exactness result can be extended to cover the case when there are monthly proportional variations in prices, and the expenditure shares have constant seasonal eﬀects in addition to linear trends; see Alterman, Diewert and Feenstra (1999, p. 65).
BASIC INDEX NUMBER THEORY tion so that it will not matter whether we use the ﬁxed base or chain principle.

15.94 Walsh (1901, p. 401; 1921a, p. 98; 1921b, p. 540) introduced the following useful variant of the circularity test: 1=P( p0, p1, q0, q1)P( p1, p2, q1, q2) .

.

.

P( pT , p0, qT , q0) (15:82) The motivation for this test is the following.
Use the bilateral index formula P( p0, p1, q0, q1) to calculate the change in prices going from period 0 to 1, use the same formula evaluated at the data corresponding to periods 1 and 2, P( p1, p2, q1, q2), to calculate the change in prices going from period 1 to 2, .

.

.
.

.

, use P(PT1, pT , qT1, qT ) to calculate the change in prices going from period T 1 to T, introduce an artiﬁcial period T+1 that has exactly the price and quantity of the initial period 0 and use P( pT , p0, qT , q0) to calculate the change in prices going from period T to 0.

Finally, multiply all of these indices together.

Since we end up where we started, the product of all of these indices should ideally be one.
Diewert (1993a, p. 40) called this test a multiperiod identity test.77 Note that if T=2 (so that the number of periods is three in total), then Walsh’s test reduces to Fisher’s (1921, p. 534; 1922, p. 64) time reversal test.78 15.95 Walsh (1901, pp.

423–433) showed how his circularity test could be used in order to evaluate how ‘‘good’’ any bilateral index number formula was.
What he did was invent artiﬁcial price and quantity data for ﬁve periods, and he added a sixth period that had the data of the ﬁrst period.

He then evaluated the right-hand side of formulae, P( p0, p1, q0, q1), and determined how far from unity the results were.

His ‘‘best’’ formulae had products that were close to one.79 equation (15.82) various for 15.96 This same framework is often used to evaluate the eﬃcacy of chained indices versus their direct coun- terparts.
Thus if the right-hand side of equation (15.82) turns out to be diﬀerent from unity, the chained indices are said to suﬀer from ‘‘chain drift’’.

If a formula does suﬀer from chain drift, it is sometimes recommended that ﬁxed base indices be used in place of chained ones.

However, this advice, if accepted, would always lead to the adoption of ﬁxed base indices, provided that index formula satisﬁes the identity test, the bilateral P( p0, p0, q0, q0)=1.
Thus it is not recommended that Walsh’s circularity test be used to decide whether ﬁxed base or chained indices should be calculated.

It is fair to use Walsh’s circularity test, as he originally used it as an 77 Walsh (1921a, p. 98) called his test the circular test, but since Fisher also used this term to describe his transitivity test deﬁned earlier by equation (15.77), it seems best to stick to Fisher’s terminology since it is well established in the literature.

78 Walsh (1921b, pp.


540–541) noted that the time reversal test was a special case of his circularity test.

79 This is essentially a variant of the methodology that Fisher (1922, p. 284) used to check how well various formulae corresponded to his version of the circularity test.

283 CONSUMER PRICE INDEX MANUAL: THEORY AND PRACTICE approximate method for deciding how ‘‘good’’ a par- ticular index number formula is.
To decide whether to chain or use ﬁxed base indices, look at how similar the observations being compared are and choose the method which will best link up the most similar observations.

15.97 Various properties, axioms or tests that an index number formula could satisfy have been intro- duced in this chapter.

In the following chapter, the test approach to index number theory is studied in a more systematic manner.
284 BASIC INDEX NUMBER THEORY Appendix 15.1 The relationship between the Paasche and Laspeyres indices Appendix 15.2 The relationship between the Lowe and Laspeyres indices 1.

Recall the notation used in paragraphs 15.11 to 15.17, above.

Deﬁne the ith relative price or price relative ri and the ith quantity relative ti as follows: ri p1 i p0 i ; ti q1 i q0 i ; i=1; .

.

.
.

.

; n (A15:1:1) Using formula (15.8) for the Laspeyres price index PL and deﬁnitions (A15.1.1), we have: n PL=P i=1 ris0 i r* (A15:1:2) i.e., we deﬁne the ‘‘average’’ price relative r* as the base period expenditure share-weighted average of the individual price relatives, ri.

2.

Using formula (15.6) for the Paasche price index PP, we 1.

Recall the notation used in paragraphs 15.33 to 15.48, above.
Deﬁne the ith relative price relating the price of com- modity i of month t to month 0, ri, and the ith quantity rela- tive, ti, relating quantity of commodity i in base year b to month 0 ti as follows: ri pt i p0 i ti qb i q0 i ; i=1; .

.

.
; n (A15:2:1) As in Appendix A15.1, the Laspeyres price index PL( p0, pt, q0) can be deﬁned as r*, the month 0 expenditure share-weighted average of the individual price relatives ri deﬁned in (A15.2.1) except that the month t price, pt i, now replaces period 1 price, p1 i , in the deﬁnition of the ith price relative ri: have: PP = n P i=1 n P m=1 n P i=1 n P m=1 i q1 p1 i mq1 p0 m = n P i=1 n P m=1 ritip0 i q0 i tmp0 mq0 m using definitions (A15:1:1) ritis0 i tms0 m 8 >>< >>: = 1 n P m=1 tms0 m n P i=1 (rir*)(tit*)s0 i 9 >>= >>; +r* (A15:1:3) using (A15.1.2) and Pn quantity relative t* is deﬁned as i=1s0 i =1 and where the ‘‘average’’ n t* P i=1 tis0 i =QL (A15:1:4) where the last equality follows using equation (15.11), the deﬁnition of the Laspeyres quantity index QL.
3.

Taking the diﬀerence between PP and PL and using equations (A15.1.2)–(A15.1.4) yields: PPPL= 1 QL n P i=1 (rir*)(tit*)s0 i (A15:1:5) Now let r and t be discrete random variables that take on the n values ri and ti respectively.

Let s0 i be the joint probability that r=ri and t=ti for i=1, .

.

.

, n and let the joint probability be 0 if r=ri and t=tj where i=j.
It can be veriﬁed that the sum- mation Pn i on the right-hand side of equa- tion (A15.1.5) is the covariance between the price relatives ri and the corresponding quantity relatives ti.

This covariance can be converted into a correlation coeﬃcient.80 If this co- variance is negative, which is the usual case in the consumer context, then PP will be less than PL.

i=1(rir*) (tit*)s0 n r* P i=1 ris0 i =PL (A15:2:2) 2.
The ‘‘average’’ quantity relative t* relating the quantities of base year b to those of month 0 is deﬁned as the month 0 expenditure share-weighted average of the individual quantity relatives ti, deﬁned in (A15.2.1): n t* P i=1 tis0 i =QL (A15:2:3) where QL=QL(q0, qb, p0) is the Laspeyres quantity index relating the quantities of month 0, q0, to those of the year b, qb, using the prices of month 0, p0, as weights.

3.
Using deﬁnition (15.26), the Lowe index comparing the prices in month t to those of month 0, using the quantity weights of the base year b, is equal to: PLo( p0, pt, qb) pt itiq0 i using (A15:2:1) n P i=1 n P i=1 8 >>< >>: 8 >>< >>: n P i=1 n P i=1 n P i=1 = = iqb pt i i qb p0 i = n P i=1 n P i=1 8 9 >>< >>= >>: >>; tip0 i q0 i i tiq0 p0 i i tiq0 p0 i i q0 p0 i , n P i=1 n P i=1 9 >>= >>; pt i tiq0 i i q0 p0 i pt i p0 i n P i=1 i q0 p0 i 1 9 >>= >>; t* using (A15:2:3) 8 >>< >>: = n P i=1 n P i=1 ritip0 i q0 i i q0 p0 i , t* using (A15:2:1) 9 >>= >>; 80 See Bortkiewicz (1923, pp.
374–375) for the ﬁrst application of this correlation coeﬃcient decomposition technique.
n P i=1 n P i=1 = = n P i=1 ritis0 i t* = (rir*)tis0 i + t* (rir*)tis0 i + ritis0 i t* n r* P i=1 t* r*tis0 i t* n P i=1 285 CONSUMER PRICE INDEX MANUAL: THEORY AND PRACTICE n P i=1 n P i=1 n P i=1 n P i=1 = = = = (rir*)tis0 i t* + r*[t*] t* (rir*)(tit*)s0 i t* (rir*)(tit*)s0 i t* (rir*)(tit*)s0 i t* using (A15:2:3) n P i=1 (rir*)t*s0 i t* n t* P i=1 ris0 i r* t* +r* +r* + + +r* n since P i=1 ris0 i =r* =PL( p0, pt, q0)+ n P i=1 (rir*)(tit*)s0 i QL(q0, qb, p0) (A15:2:4) since using (A15.2.2), r* equals the Laspeyres price index, PL( p0, pt, q0), and using (A15.2.3), t* equals the Laspeyres quantity index, QL(q0, qb, p0).
Thus equation (A15.2.4) tells us that the Lowe price index using the quantities of year b as weights, PLo( p0, pt, qb), is equal to the usual Laspeyres index using the quantities of month 0 as weights, PL( p0, pt, q0), plus a covariance term Pn i between the price rela- tives ri pt i and the quantity relatives ti qb i , divided by the Laspeyres quantity index QL(q0, qb, p0) between month 0 and base year b. i=1(rir*)(tit*)s0 i =q0 i =p0 Appendix 15.3 The relationship between the Young index and its time antithesis 1.
Recall that the direct Young index, PY(p0, pt, sb), was antithesis, * (p0, pt, sb), was deﬁned by equation (15.52).

Deﬁne the ith deﬁned by PY relative price between months 0 and t as equation (15.48) and its time ri pt i =p0 i ; i=1, .

.

.

, n (A15:3:1) and deﬁne the weighted average (using the base year weights sb i ) of the ri as n r* P i=1 sb i ri (A15:3:2) which turns out to equal the direct Young index, PY(p0, pt, sb).
Deﬁne the deviation ei of ri from their weighted average r* using the following equations: ri=r*(1+ei); i=1, .

.

.

, n (A15:3:3) If equation (A15.3.3) is substituted into equation (A15.3.2), the following equation is obtained: sb i r*(1+ei) n r* P i=1 n =r*+r* P i=1 sb i ei n sinceP i=1 sb i =1 (A15:3:4) n e* P i=1 sb i ei=0 (A15:3:5) Thus the weighted mean e* of the deviations ei equals 0.

2.
2.

The direct Young index, PY(p0, pt, sb), and its time Y (p0, pt, sb), can be written as functions of r*, i and the deviations of the price relatives ei as antithesis, P* the weights sb follows: PY ( p0, pt, sb)=r* n (A15:3:6) 1 Y (p0, pt, sb)= P P* i fr*(1+ei)g1 sb i=1 n =r* P i=1 i (1+ei)1 sb 1 (A15:3:7) 3.

Now regard P* deviations, e:[e1, .

.

.


, en], say P* series approximation to P* by the following expression:81 Y ( p0, pt, sb) as a function of the vector of Y (e).
The second-order Taylor Y (e) around the point e=0n is given Y (e) r*+r* P P* n i=1 n i ei+r* P sb " i=1 n P j=1 n =r*+r*0+r* P i=1 sb i n P j=1 sb j ej i sb sb j eiejr* P i [ei]2 sb n i=1 # n eir* P i=1 i [eie*]2 sb using (A15:3:5) 81 This type of second-order approximation is attributable to Dale´ n (1992; 143) for the case r*=1 and to Diewert (1995a, p. 29) for the case of a general r*.
286 n =r*+r* P i=1 n i [0]eir* P sb i=1 i [eie*]2 using (A15:3:5) sb =PY ( p0, pt, sb)PY (p0, pt, sb)P n i [eie*]2 sb i=1 using (A15:3:6) =PY ( p0, pt, sb)PY (p0, pt, sb)Var e where the weighted sample variance of the vector e of price deviations is deﬁned as n Var e P i=1 i [eie*]2 sb (A15:3:9) 4.
Rearranging equation (A15.3.8) gives the following approximate relationship between the direct Young index PY(p0, pt, sb) and its time antithesis P* Y (p0, pt, sb), to the accu- racy of a second-order Taylor series approximation about a price point where the month t price vector is proportional to the month 0 price vector: PY ( p0, pt, sb) P* Y ( p0, pt, sb)+PY ( p0, pt, sb)Var e (A15:3:10) Thus, to the accuracy of a second-order approximation, the direct Young index will exceed its time antithesis by a term equal to the direct Young index times the weighted variance of the deviations of the price relatives from their weighted mean.
Thus the bigger is the dispersion in relative prices, the more the direct Young index will exceed its time antithesis.

BASIC INDEX NUMBER THEORY Appendix 15.4 The relationship between the Divisia and economic approaches 1.

Divisia’s approach to index number theory relied on the theory of diﬀerentiation.

Thus it does not appear to have any connection with economic theory.
However, starting with Ville (1946), a number of economists82 have established that the Divisia price and quantity indices do have a connection with the economic approach to index number theory.

This con- nection is outlined in this appendix.

2.

The economic approach to the determination of the price level and the quantity level is ﬁrst outlined.

The partic- ular economic approach that is used here is attributable to Shephard (1953; 1970), Samuelson (1953) and Samuelson and Swamy (1974).

3.
3.

It is assumed that ‘‘the’’ consumer has well-deﬁned preferences over diﬀerent combinations of the n consumer commodities or items.

Each combination of items can be represented by a positive vector q:[q1, .

.

.

, qn].
.

.

, qn].

The con- sumer’s preferences over alternative possible consumption vectors q are assumed to be representable by a continuous, non-decreasing and concave utility function f. It is further assumed that the consumer minimizes the cost of achieving the period t utility level ut:f (qt) for periods t=0,1, .

.

.
.

.

, T. Thus it is assumed that the observed period t consumption vector qt solves the following period t cost minimization problem: C(ut, pt) minq n P i=1 i qi : f (q)=ut=f (qt) pt n =P i=1 i qt pt i; t=0, 1, .

.

.

, T (A15:4:1) The period t price vector for the n commodities under con- sideration that the consumer faces is pt.

Note that the solution to the period t cost or expenditure minimization problem deﬁnes the consumer’s cost function, C(ut, pt).

4.
4.

An additional regularity condition is placed on the consumer’s utility function f. It is assumed that f is (positively) linearly homogeneous for strictly positive quantity vectors.

Under this assumption, the consumer’s expenditure or cost function, C(u, p), decomposes into uc( p) where c( p) is the consumer’s unit cost function.83 The following equation is obtained: n P i=1 i qt pt i=c( pt)f (qt) for t=0, 1, .

.

.
.

.

, T (A15:4:2) iqt i=1pt Thus the period t total expenditure on the n commodities in the aggregate, Pn i , decomposes into the product of two terms, c( pt) f(qt).

The period t unit cost, c( pt), can be identiﬁed as the period t price level Pt and the period t level of utility, f(qt), can be identiﬁed as the period t quantity level Qt.

5.
5.

The economic price level for period t, Pt:c( pt), deﬁned in the previous paragraph, is now related to the Divisia price level for time t, P(t), that was implicitly deﬁned by the diﬀer- ential equation (15.67).

As in paragraphs 15.65 to 15.71, think of the prices as being continuous, diﬀerentiable functions of 82 See for example Malmquist (1953, p. 227), Wold (1953, pp.
134– 147), Solow (1957), Jorgenson and Griliches (1967) and Hulten (1973), and see Balk (2000a) for a recent survey of work on Divisia price and quantity indices.

83 See Diewert (1993b, pp.120–121) for material on unit cost functions.

This material will also be covered in Chapter 17.

287 CONSUMER PRICE INDEX MANUAL: THEORY AND PRACTICE time, pi(t) say, for i:1, .

.

.
.

.

, n. Thus the unit cost function can be regarded as a function of time t as well; i.e., deﬁne the unit cost function as a function of t as P0(t) P(t) = c*(t) c[ p1(t), p2(t), .

.

.

, pn(t)] (A15:4:3) 6.

Assuming that the ﬁrst-order partial derivatives of the unit cost function c(p) exist, calculate the logarithmic deriva- tive of c*(t) as follows: = = dc*(t) dt d ln c*(t) dt = 1 c*(t) n P i=1 ci[ p1(t), p2(t), .

.

.

, pn(t)]p0 i(t) c[ p1(t), p2(t), .

.

.


, pn(t)] (A15:4:4) where ci[p1(t), p2(t), .

.

.

, pN(t)]:@c[p1(t), p2(t), .

.

.

, pn(t)]/@pi is the partial derivative of the unit cost function with respect to the ith price, pi, and p0 i(t) dpi(t)=dt is the time derivative of the ith price function, pi(t).

Using Shephard’s (1953, p. 11) Lemma, the consumer’s cost minimizing demand for com- modity i at time t is: qi(t)=u(t)cifp1(t), p2(t), .

.

.

, pn(t)g for i=1, .

.

.


, n (A15:4:5) where the utility level at time t is u(t)=f [q1(t), q2(t), .

.

.

, qn(t)].

The continuous time counterpart to equations (A15.4.2) above is that total expenditure at time t is equal to total cost at time t which in turn is equal to the utility level, u(t), times the period t unit cost, c*(t): n P i=1 pi(t)qi(t)=u(t)c*(t)=u(t)cf p1(t), p2(t), .

.

.

, pn(t)g (A15:4:6) 7.


The logarithmic derivative of the Divisia price level P(t) can be written as (recall equation (15.67) above): n P i=1 n P i=1 n P i=1 n P i=1 n P i=1 p0 i(t)qi(t) u(t)c*(t) = p0 i(t)qi(t) pi(t)qi(t) using (A15:4:6) p0 i(t)[u(t)cfp1(t), p2(t), .

.

.

, pn(t)g] u(t)c*(t) using (A15:4:5) cifp1(t), p2(t), .

.

.
.

.

, pn(t)gp0 i(t) c*(t) = 1 c*(t) dc*(t) dt using (A15:4:4) c*0(t) c*(t) (A15:4:7) the above continuous Thus under time cost-minimizing assumptions, the Divisia price level, P(t), is essentially equal to the time t prices, the unit cost c*(t):c[p1(t), p2(t), .

.

.

, pN(t)].

function evaluated at 8.

If the Divisia price level P(t) is set equal to the unit cost function c*(t):c[p1(t), p2(t), .

.

.
.

.

, pN(t)], then from equation (A15.4.2), it follows that the Divisia quantity level Q(t) deﬁned by equation (15.68) will equal the consumer’s utility function regarded as a function of time, f *(t):f [q1(t), .

.

.

, qn(t)].
Thus, under the assumption that the consumer is continuously minimizing the cost of achieving a given utility level where the utility or preference function is linearly homogeneous, it has been shown that the Divisia price and quantity levels P(t) and Q(t), deﬁned implicitly by the diﬀerential equations (15.67) and (15.68), are essentially equal to the consumer’s unit cost function c*(t) and utility function f *(t) respectively.84 These are rather remarkable equalities since in principle, given the functions of time, pi(t) and qi(t), the diﬀerential equations that deﬁne the Divisia price and quantity indices can be solved numerically and hence P(t) and Q(t) are in principle observable (up to some normalizing constants).
9.

For more on the Divisia approach to index number theory, see Vogt (1977; 1978) and Balk (2000a).

An alternative approach to Divisia indices using line integrals may be found in the forthcoming companion volume Producer price index manual (Eurostat et al., 2004).

288 84 Obviously, the scale of the utility and cost functions are not uniquely determined by the diﬀerential equations (15.62) and (15.63).
THE AXIOMATIC AND STOCHASTIC APPROACHES TO INDEX NUMBER THEORY 16 Introduction 16.1 As was seen in Chapter 15, it is useful to be able to evaluate various index number formulae that have been proposed in terms of their properties.

If a formula turns out to have rather undesirable properties, this casts doubts on its suitability as an index that could be used by a statistical agency as a target index.
Looking at the mathematical properties of index number formulae leads to the test or axiomatic approach to index number theory.

In this approach, desirable properties for an index number formula are proposed, and it is then attempted to determine whether any formula is con- sistent with these properties or tests.

An ideal outcome is the situation where the proposed tests are both desirable and completely determine the functional form for the formula.
16.2 The axiomatic approach to index number the- ory is not completely straightforward, since choices have to be made in two dimensions: The index number framework must be determined.

 Once the framework has been decided upon, it must be decided what tests or properties should be imposed on the index number.
The second point is straightforward: diﬀerent price statisticians may have diﬀerent ideas about which tests are important, and alternative sets of axioms can lead to alternative ‘‘best’’ index number functional forms.

This point must be kept in mind while reading this chapter, since there is no universal agreement on what the ‘‘best’’ set of ‘‘reasonable’’ axioms is.

Hence the axiomatic approach can lead to more than one best index number formula.
16.3 The ﬁrst point about choices listed above requires further discussion.

In the previous chapter, for the most part, the focus was on bilateral index number theory; i.e., it was assumed that prices and quantities for the same n commodities were given for two periods and the object of the index number formula was to compare the overall level of prices in one period with the other period.
In this framework, both sets of price and quantity vectors were regarded as variables which could be independently varied so that, for example, variations in the prices of one period did not aﬀect the prices of the other period or the quantities in either period.

The emphasis was on comparing the overall cost of a ﬁxed basket of quantities in the two periods or taking averages of such ﬁxed basket indices.

This is an example of an index number framework.


16.4 However, other index number frameworks are possible.

For example, instead of decomposing a value ratio into a term that represents price change between the two periods times another term that represents quantity change, an attempt could be made to decom- pose a value aggregate for one period into a single number that represents the price level in the period times another number that represents the quantity level in the period.
In the ﬁrst variant of this approach, the price index number is supposed to be a function of the n commodity prices pertaining to that aggregate in the period under consideration, while the quantity index number is supposed to be a function of the n commodity quantities pertaining to the aggregate in the period.
The resulting price index function was called an absolute index number by Frisch (1930, p. 397), a price level by Eichhorn (1978, p. 141) and a unilateral price index by Anderson, Jones and Nesmith (1997, p. 75).
In a second variant of this approach, the price and quantity func- tions are allowed to depend on both the price and quantity vectors pertaining to the period under con- sideration.1 These two variants of unilateral index number theory will be considered in paragraphs 16.11 to 16.29.2 16.5 The remaining approaches in this chapter are largely bilateral approaches; i.e., the prices and quan- tities in an aggregate are compared for two periods.
In paragraphs 16.30 to 16.73 and 16.94 to 16.129, the value ratio decomposition approach is taken.3 In paragraphs 16.30 to 16.73, the bilateral price and quantity indices, P( p0, p1, q0, q1) and Q( p0, p1, q0, q1), are regarded as functions of the price vectors pertaining to the two periods, p0 and p1, and the two quantity vectors, q0 and q1.
Not only do the axioms or tests that are placed on the price index P( p0, p1, q0, q1) reﬂect ‘‘reasonable’’ price index properties, but some tests have their origin as ‘‘reasonable’’ tests on the quan- tity index Q( p0, p1, q0, q1).

The approach in paragraphs 16.30 to 16.73 simultaneously determines the ‘‘best’’ price and quantity indices.
16.6 In paragraphs 16.74 to 16.93, attention is shif- ted to the price ratios for the n commodities between 1 Eichhorn (1978, p. 144) and Diewert (1993d, p. 9) considered this approach.

2 In these unilateral index number approaches, the price and quan- tity vectors are allowed to vary independently.

In yet another index number framework, prices are allowed to vary freely but quantities are regarded as functions of the prices.
This leads to the economic approach to index number theory, which is considered brieﬂy in Appendix 15.4 of Chapter 15, and in more depth in Chapters 17 and 18.

3 Recall paragraphs 15.7 to 15.17 of Chapter 15 for an explanation of this approach.

289 CONSUMER PRICE INDEX: THEORY AND PRACTICE periods 0 and 1, ri p1 i =p0 for i=1, .

.

.
.

.

, n. In the i unweighted stochastic approach to index number theory, the price index is regarded as an evenly weighted average of the n price relatives or ratios, ri.

Carli (1764) and Jevons (1863; 1865) were the earlier pio- neers in this approach to index number theory, with Carli using the arithmetic average of the price relatives and Jevons endorsing the geometric average (but also considering the harmonic average).
This approach to index number theory will be covered in paragraphs 16.74 to 16.79.

This approach is consistent with a statistical approach that regards each price ratio ri as a random variable with mean equal to the underlying price index.

16.7 A major problem with the unweighted aver- age of price relatives approach to index number the- ory is that this approach does not take into account the economic importance of the individual commod- ities in the aggregate.
Young (1812) did advocate some form of rough weighting of the price relatives according to their relative value over the period being considered, but the precise form of the required value weighting was not indicated.4 It was Walsh (1901, pp.

83–121; 1921a, pp.
81–90), however, who stressed the importance of weighting the individual price ratios, where the weights are functions of the associated values for the commodities in each period and each period is to be treated symmetrically in the resulting formula: What we are seeking is to average the variations in the exchange value of one given total sum of money in rela- tion to the several classes of goods, to which several variations [price ratios] must be assigned weights pro- portional to the relative sizes of the classes.
Hence the relative sizes of the classes at both the periods must be considered (Walsh (1901, p. 104)).

Commodities are to be weighted according to their importance, or their full values.

But the problem of axiometry always involves at least two periods.

There is a ﬁrst period and there is a second period which is com- pared with it.

Price variations5 have taken place between the two, and these are to be averaged to get the amount of their variation as a whole.
But the weights of the com- modities at the second period are apt to be diﬀerent from their weights at the ﬁrst period.

Which weights, then, are the right ones – those of the ﬁrst period or those of the second?

Or should there be a combination of the two sets?

There is no reason for preferring either the ﬁrst or the second.

Then the combination of both would seem to be the proper answer.

And this combination itself involves an averaging of the weights of the two periods (Walsh (1921a, p. 90)).
4 Walsh (1901, p. 84) refers to Young’s contributions as follows: Still, although few of the practical investigators have actually employed anything but even weighting, they have almost always recognized the theoretical need of allowing for the relative importance of the diﬀerent classes ever since this need was ﬁrst pointed out, near the commence- .

.

.

Arthur Young ment of the century just ended, by Arthur Young.

advised simply that the classes should be weighted according to their importance.
5 A price variation is a price ratio or price relative in Walsh’s termi- nology.

290 16.8 Thus Walsh was the ﬁrst to examine in some detail the rather intricate problems6 involved in deciding how to weight the price relatives pertaining to an aggregate, taking into account the economic importance of the commodities in the two periods being considered.
Note that the type of index number formula that Walsh was considering was of the form P(r, v0, v1), where r is the vector of price relatives which has ith component i and vt is the period t value vector which has ri=p1 ith component vt i for t=0, 1.

His suggested solu- tion to this weighting problem was not completely satisfactory but he did at least suggest a very useful framework for a price index, as a value-weighted aver- age of the n price relatives.
The ﬁrst satisfactory solu- tion to the weighting problem was obtained by Theil (1967, pp.

136–137) and his solution is explained in paragraphs 16.79 to 16.93. i=pt i =p0 iqt 16.9 It can be seen that one of Walsh’s approaches to index number theory7 was an attempt to determine the ‘‘best’’ weighted average of the price relatives, ri.

This is equivalent to using an axiomatic approach to try to determine the ‘‘best’’ index of the form P(r, v0, v1).
This approach is considered in paragraphs 16.94 to 16.129.8 16.10 The Young and Lowe indices, discussed in Chapter 15, do not ﬁt precisely into the bilateral frame- work since the value or quantity weights used in these indices do not necessarily correspond to the values or quantities that pertain to either of the periods that correspond to the price vectors p0 and p1.

The axio- matic properties of these two indices with respect to their price variables are studied in paragraphs 16.130 to 16.134.
6 Walsh (1901, pp.

104–105) realized that it would not do simply to take the arithmetic average of the values in the two periods, [v0 i ]=2, as the ‘‘correct’’ weight for the ith price relative ri since, in a period of rapid inﬂation, this would give too much importance to the period that had the highest prices and he wanted to treat each period symme- trically: i +v1 But such an operation is manifestly wrong.
In the ﬁrst place, the sizes of the classes at each period are reckoned in the money of the period, and if it happens that the exchange value of money has fallen, or prices in general have risen, greater inﬂuence upon the result would be given to the weighting of the second period; or if prices in general have fallen, greater inﬂuence would be given to the weighting of the second period.
Or in a comparison between two countries greater inﬂuence would be given to the weighting of the country with the higher level of prices.

But it is plain that the one period, or the one country, is as important, in our comparison between them, as the other, and the weighting in the averaging of their weights should really be even.
i +s1 However, Walsh was unable to come up with Theil’s (1967) solution to the weighting problem, which was to use the average expenditure share [s0 i ]=2, as the ‘‘correct’’ weight for the ith price relative in the context of using a weighted geometric mean of the price relatives.

7 Walsh also considered basket-type approaches to index number theory, as was seen in Chapter 15.
8 In paragraphs 16.94 to 16.129, rather than starting with indices of the form P(r, v0, v1), indices of the form P( p0, p1, v0, v1) are considered.

However, if the test of invariance to changes in the units of measure- ment is imposed on this index, it is equivalent to studying indices of the form P(r, v0, v1).

Vartia (1976) also used a variation of this approach to index number theory.
THE AXIOMATIC AND STOCHASTIC APPROACHES TO INDEX NUMBER THEORY The levels approach to index number theory An axiomatic approach to unilateral price indices i and qt i be deﬁned as a unit value; 16.11 Denote the price and quantity of commodity n in period t by pt i respectively for i=1, 2, .

.

.

, n and t=0, 1, .

.

.


, T. The variable qt i is interpreted as the total amount of commodity i transacted within period t. In order to conserve the value of transactions, it is necessary that pt i.e., pt i must be equal to the value of transactions in com- modity i for period t divided by the total quantity transacted, qt i.
In principle, the period of time should be chosen so that variations in commodity prices within a period are very small compared to their variations between periods.9 For t=0, 1, .

.

.

, T, and i=1, .

.

.

, n, deﬁne the value of transactions in com- modity i as vt i and deﬁne the total value of transactions in period t as: i pt iqt n V t P i=1 n i=P vt iqt pt i i=1 t=0, 1, .

.

.
.

.

, T (16:1) 16.12 Using the above notation, the following levels version of the index number problem is deﬁned as follows: for t=0, 1, .

.

.

, T, ﬁnd scalar numbers Pt and Qt such that V t=PtQt t=0, 1, .

.

.

, T (16:2) The number Pt is interpreted as an aggregate period t price level, while the number Qt is interpreted as an aggregate period t quantity level.
The aggregate price level Pt is allowed to be a function of the period t price vector, pt, while the aggregate period t quantity level Qt is allowed to be a function of the period t 9 This treatment of prices as unit values over time follows Walsh (1901, p. 96; 1921a, p. 88) and Fisher (1922, p. 318).
Fisher and Hicks both had the idea that the length of the period should be short enough so that variations in price within the period could be ignored, as the following quotations indicate: Throughout this book ‘‘the price’’ of any commodity or ‘‘the quantity’’ of it for any one year was assumed given.

But what is such a price or quantity?

Sometimes it is a single quotation for January 1 or July 1, but usually it is an average of several quotations scattered throughout the year.
The question arises: On what principle should this average be con- structed?

The practical answer is any kind of average since, ordinarily, the variations during a year, so far, at least, as prices are concerned, are too little to make any perceptible diﬀerence in the result, whatever kind of average is used.

Otherwise, there would be ground for subdividing the year into quarters or months until we reach a small enough period to be considered practically a point.
The quantities sold will, of course, vary widely.

What is needed is their sum for the year (which, of course, is the same thing as the simple arithmetic average of the per annum rates for the separate months or other subdivisions).

In short, the simple arithmetic average, both of prices and of quantities, may be used.

Or, if it is worth while to put any ﬁner point on it, we may take the weighted arithmetic average for the prices, the weights being the quantities sold (Fisher (1922, p. 318)).
I shall deﬁne a week as that period of time during which variations in prices can be neglected.

For theoretical purposes this means that prices will be supposed to change, not continuously, but at short intervals.

The calendar length of the week is of course quite arbitrary; by taking it to be very short, our theoretical scheme can be ﬁtted as closely as we like to that ceaseless oscillation which is a characteristic of prices in certain markets (Hicks (1946, p. 122)).
quantity vector, qt; hence: Pt=c( pt) and Qt=f (qt) t=0, 1, .

.

.

, T (16:3) 16.13 The functions c and f are to be determined somehow.

Note that equation (16.3) requires that the functional forms for the price aggregation function c and for the quantity aggregation function f be inde- pendent of time.

This is a reasonable requirement since there is no reason to change the method of aggregation as time changes.
16.14 Substituting equations (16.3) and (16.2) into equation (16.1) and dropping the superscripts t means that c and f must satisfy the following functional equa- tion for all strictly positive price and quantity vectors: n c( p) f (q)= P i=1 piqi for all pi > 0 and for all qi > 0 (16:4) 16.15 It is natural to assume that the functions c( p) and f(q) are positive if all prices and quantities are positive: c( p1, .

.

.

, pn) > 0; f (q1, .

.

.


, qn) > 0 if all pi > 0 and all qi > 0 (16:5) 16.16 Let 1n denote an n-dimensional vector of ones.
Then (16.5) implies that when p=1n, c(1n) is a positive number, a for example, and when q=1n, then f(1n) is also a positive number, b for example; i.e., (16.5) implies that c and f satisfy: c(1n)=a > 0; f (1n)=b > 0 (16:6) 16.17 Let p=1n and substitute the ﬁrst equation in (16.6) into equation (16.4) in order to obtain the fol- lowing equation: n f (q)= P i=1 qi a for all qi > 0 (16:7) 16.18 Now let q=1n and substitute the second equation in (16.6) into equation (16.4) in order to obtain the following equation: n c( p)=P i=1 pi b for all pi > 0 (16:8) 16.19 Finally substitute equations (16.7) and (16.8) into the left-hand side of equation (16.4) to obtain the following equation: pi b n n P P i=1 i=1 and for all qi > 0 qi a n =P i=1 piqi for all pi > 0 (16:9) If n is greater than one, it is obvious that equation (16.9) cannot be satisﬁed for all strictly positive p and q vec- tors.
Thus if the number of commodities n exceeds one, then there do not exist any functions c and f that satisfy equations (16.4) and (16.5).10 10 Eichhorn (1978, p. 144) established this result.
291 CONSUMER PRICE INDEX: THEORY AND PRACTICE 16.20 Thus this levels test approach to index number theory comes to an abrupt halt; it is fruitless to look for price and quantity level functions, Pt=c( pt) and Qt=f(qt), that satisfy equations (16.2) or (16.4) and also satisfy the very reasonable positivity requirements (16.5).
16.21 Note that the levels price index function, c( pt), did not depend on the corresponding quantity vector qt and the levels quantity index function, f (qt), did not depend on the price vector pt.

Perhaps this is the reason for the rather negative result obtained above.

Hence, in the next section, the price and quantity functions are allowed to be functions of both pt and qt.
A second axiomatic approach to unilateral price indices 16.22 In this section, the goal is to ﬁnd functions of 2n variables, c( p, q) and f( p, q), such that the following counterpart to equation (16.4) holds: n c( p, q)f ( p, q)= P i=1 piqi for all pi > 0 and for all qi > 0 (16:10) 16.23 Again, it is natural to assume that the func- tions c( p, q) and f( p, q) are positive if all prices and quantities are positive: c( p1, .

.

.

, pn; q1, .

.

.

, qn) > 0; f ( p1, .

.

.

, pn; q1, .

.

.


, qn) > 0 if all pi > 0 and all qi > 0 (16:11) 16.24 The present framework does not distinguish between the functions c and f, so it is necessary to require that these functions satisfy some ‘‘reasonable’’ properties.
The ﬁrst property imposed on c is that this function be homogeneous of degree one in its price components: c(lp, q)=lc( p, q) for all l > 0 (16:12) Thus, if all prices are multiplied by the positive number l, then the resulting price index is l times the initial price index.
A similar linear homogeneity property is imposed on the quantity index f; i.e., f is to be homogeneous of degree one in its quantity components: f ( p, lq)=l f ( p, q) for all l > 0 (16:13) 16.25 Note that properties (16.10), (16.11) and (16.13) imply that the price index c( p, q) has the fol- lowing homogeneity property with respect to the com- ponents of q: n c( p, lq)=P i=1 n =P i=1 n =P i=1 pilqi f ( p, lq) pilqi lf ( p, q) piqi f ( p, q) where l > 0 using (16:3) =c( p, q) using (16:10) and (16:11) (16:14) Thus c( p, q) is homogeneous of degree zero in its q components.
292 16.26 A ﬁnal property that is imposed on the levels price index c( p, q) is the following one.

Let the positive numbers di be given.
Then it is asked that the price index be invariant to changes in the units of measurement for the n commodities so that the function c( p, q) has the following property: c(d1p1, ...,dnpn;q1=d1, ...,qn=dn) =c(p1, ...,pn; q1, ...,qn) (16:15) 16.27 It is now possible to show that properties (16.10), (16.11), (16.12), (16.14) and (16.15) on the price levels function c( p, q) are inconsistent; i.e., there does not exist a function of 2n variables c( p, q) that satisﬁes these very reasonable properties.11 16.28 To see why this is so, apply the equation (16.15), setting di=qi for each i, to obtain the following equation: c( p1, .
.

.

, pn; q1, .

.

.

, qn)=c( p1q1, .

.

.

, pnqn;1, .

.

.

, 1) (16:16) If c( p, q) satisﬁes the linear homogeneity property (16.12) so that c(lp, q)=lc( p, q), then equation (16.16) implies that c( p, q) is also linearly homogeneous in q so that c( p, lq)=lc( p, q).

But this last equation contradicts equation (16.14), which establishes the impossibility result.
16.29 The rather negative results obtained in para- graphs 16.13 to 16.21 indicate that it is fruitless to pursue the axiomatic approach to the determination of price and quantity levels, where both the price and quantity vector are regarded as independent variables.12 Hence, in the following sections of this chapter, the axiomatic approach to the determination of a bilateral price index of the form P( p0, p1, q0, q1) will be pursued.
The ﬁrst axiomatic approach to bilateral price indices Bilateral indices and some early tests 16.30 In this section, the strategy will be to assume that the bilateral price index formula, P( p0, p1, q0, q1), satisﬁes a suﬃcient number of ‘‘reasonable’’ tests or properties so that the functional form for P is deter- mined.13 The word ‘‘bilateral’’14 refers to the assump- function P depends only on the tion that data pertaining to the two situations or periods being i.e., P is regarded as a function of the compared; two sets of price and quantity vectors, p0, p1, q0, q1, that the 11 This proposition is due to Diewert (1993d, p. 9), but his proof is an adaptation of a closely related result due to Eichhorn (1978, pp.
144–145).

12 Recall that in the economic approach, the price vector p is allowed to vary independently, but the corresponding quantity vector q is regar- ded as being determined by p. 13 Much of the material in this section is drawn from sections 2 and 3 of Diewert (1992a).

For more recent surveys of the axiomatic approach see Balk (1995) and von Auer (2001).

14 Multilateral index number theory refers to the case where there are more than two situations whose prices and quantities need to be aggregated.
THE AXIOMATIC AND STOCHASTIC APPROACHES TO INDEX NUMBER THEORY n=p0 n. 1, .

.

.
, p1 are to be aggregated into a single number that sum- marizes the overall change in the n price ratios, 1=p0 p1 16.31 In this section, the value ratio decomposition approach to index number theory will be taken; i.e., there along with the price is a companion quantity index Q(p0, p1, q0, q1) such that the product of these two indices equals the value ratio between the two periods.15 Thus, throughout this sec- tion, it is assumed that P and Q satisfy the following product test: index P( p0, p1, q0, q1), T3: Identity or constant prices test:20 P( p, p, q0, q1)=1 That is, if the price of every good is identical during the two periods, then the price index should equal unity, no matter what the quantity vectors are.
The controversial aspect of this test is that the two quantity vectors are allowed to be diﬀerent in the test.21 T4: Fixed basket or constant quantities test:22 V 1=V 0=P( p0, p1, q0, q1) Q( p0, p1, q0, q1): (16:17) P( p0, p1, q, q)= n P i=1 n P i=1 p1 i qi p0 i qi The period t values, V t, for t=0, 1 are deﬁned by equation (16.1).

As soon as the functional form for the price index P is determined, then equation (16.17) can be used to determine the functional form for the quantity index Q.
A further advantage of assuming that the product test holds is that, if a reasonable test is imposed on the quantity index Q, then equation (16.17) can be used to translate this test on the quantity index into a corresponding test on the price index P.16 16.32 If n=1, so that there is only one price and quantity to be aggregated, then a natural candidate for P is p1 1=p0 1, the single price ratio, and a natural candidate for Q is q1 1=q0 1, the single quantity ratio.
When the number of commodities or items to be aggregated is greater than 1, then what index number theorists have done over the years is propose properties or tests that the price index P should satisfy.

These properties are generally multi- dimensional analogues to the one good price index for- mula, p1 1.

Below, some 20 tests are listed that turn out to characterize the Fisher ideal price index.
1=p0 16.33 It will be assumed that every component of each price and quantity vector is positive; i.e., pt 0n and qt 0n 17 for t=0, 1.

If it is desired to set q0=q1, the common quantity vector is denoted by q; if it is desired to set p0=p1, the common price vector is de- noted by p. 16.34 The ﬁrst two tests, denoted T1 and T2, are not very controversial, so they will not be discussed in detail.
T1: Positivity:18 P( p0, p1, q0, q1) > 0 T2: Continuity:19 P( p0, p1, q0, q1) is a continuous function of its arguments 16.35 The next two tests, T3 and T4, are somewhat more controversial.

15 See paragraphs 15.7 to 15.25 of Chapter 15 for more on this approach, which was initially due to Fisher (1911, p. 403; 1922).

16 This observation was ﬁrst made by Fisher (1911, pp.

400–406), and the idea was pursued by Vogt (1980) and Diewert (1992a).
17 The notation q 0n means that each component of the vector q is positive; q 0n means each component of q is non-negative and q>0n means q 0n and q=0n.

18 Eichhorn and Voeller (1976, p. 23) suggested this test.

19 Fisher (1922, pp.

207–215) informally suggested the essence of this test.

i qi.
i qi.

i=1p0 i=1p1 That is, if quantities are constant during the two periods so that q0=q1:q, then the price index should equal the expenditure on the constant basket in period 1, Pn i qi, divided by the expenditure on the basket in period 0, Pn 16.36 If the price index P satisﬁes Test T4 and P and Q jointly satisfy the product test (16.17) above, then it is easy to show23 that Q must satisfy the identity test Q( p0, p1, q, q)=1 for all strictly positive vectors p0, p1, q.
This constant quantities test for Q is also somewhat controversial since p0 and p1 are allowed to be diﬀerent.

Homogeneity tests 16.37 The following four tests, T5–T8, restrict the behaviour of the price index P as the scale of any one of the four vectors p0, p1, q0, q1 changes.
T5: Proportionality in current prices:24 P( p0, lp1, q0, q1)=lP( p0, p1, q0, q1) for l > 0 That is, if all period 1 prices are multiplied by the positive number l, then the new price index is l times 20 Laspeyres (1871, p. 308), Walsh (1901, p. 308) and Eichhorn and Voeller (1976, p. 24) have all suggested this test.

Laspeyres came up with this test or property to discredit the ratio of unit values index of Drobisch (1871a), which does not satisfy this test.
This test is also a special case of Fisher’s (1911, pp.

409–410) price proportionality test.

21 Usually, economists assume that, given a price vector p, the corre- sponding quantity vector q is uniquely determined.

Here, the same price vector is used but the corresponding quantity vectors are allowed to be diﬀerent.
22 The origins of this test go back at least 200 years to the Massa- chusetts legislature, which used a constant basket of goods to index the pay of Massachusetts soldiers ﬁghting in the American Revolution; see Willard Fisher (1913).

Other researchers who have suggested the test over the years include: Lowe (1823, Appendix, p. 95), Scrope (1833, p. 406), Jevons (1865), Sidgwick (1883, pp.
67–68), Edgeworth (1925, p. 215) originally published in 1887, Marshall (1887, p. 363), Pierson (1895, p. 332), Walsh (1901, p. 540; 1921b, pp.

543–544), and Bowley (1901, p. 227).

Vogt and Barta (1997, p. 49) correctly observe that this test is a special case of Fisher’s (1911, p. 411) proportionality test for quantity indexes which Fisher (1911, p. 405) translated into a test for the price index using the product test (15.3).

23 See Vogt (1980, p. 70).


24 This test was proposed by Walsh (1901, p. 385), Eichhorn and Voeller (1976, p. 24) and Vogt (1980, p. 68).

293 CONSUMER PRICE INDEX: THEORY AND PRACTICE the old price index.

Put another way, the price index function P( p0, p1, q0, q1) is (positively) homogeneous of degree one in the components of the period 1 price vector p1.

Most index number theorists regard this property as a very fundamental one that the index number formula should satisfy.
16.38 Walsh (1901) and Fisher (1911, p. 418; 1922, p. 420) proposed the related proportionality test P( p, lp, q0, q1)=l.

This last test is a combination of T3 and T5; in fact Walsh (1901, p. 385) noted that this last test implies the identity test, T3.
16.39 In the next test, instead of multiplying all period 1 prices by the same number, all period 0 prices are multiplied by the number l. T6: Inverse proportionality in base period prices:25 P(lp0, p1, q0, q1)=l1P( p0, p1, q0, q1) for l > 0 That is, if all period 0 prices are multiplied by the positive number l, then the new price index is 1/l times the old price index.
Put another way, the price index function P( p0, p1, q0, q1) is (positively) homogeneous of degree minus one in the components of the period 0 price vector p0.

16.40 The following two homogeneity tests can also be regarded as invariance tests.

T7: Invariance to proportional changes in current quantities: P( p0, p1, q0, lq1)=P( p0, p1, q0, q1) for all l > 0 (1980, p. 70) was the ﬁrst That is, if current period quantities are all multiplied by the number l, then the price index remains unchanged.
Put another way, the price index function P( p0, p1, q0, q1) is (positively) homogeneous of degree zero in the components of the period 1 quantity vector q1.

to propose this Vogt test26 and his derivation of the test is of some interest.

Suppose the quantity index Q satisﬁes the quantity analogue to the price test T5; i.e., suppose Q satisﬁes Q( p0, p1, q0, lq1)=lQ( p0, p1, q0, q1) for l>0.

Then, using the product test (16.17), it can be seen that P must satisfy T7.
T8: Invariance to proportional changes in base quantities:27 P( p0, p1, lq0, q1)=P( p0, p1, q0, q1) for all l > 0 That is, if base period quantities are all multiplied by the number l, then the price index remains unchanged.

Put another way, the price index function P( p0, p1, q0, q1) is (positively) homogeneous of degree zero in the compo- nents of the period 0 quantity vector q0.
If the quantity index Q satisﬁes the following counterpart to T8: Q( p0, p1, lq0, q1)=l1Q( p0, p1, q0, q1) for all l>0, then using 25 Eichhorn and Voeller (1976, p. 28) suggested this test.

26 Fisher lq0)=P( p0, p1, q0, q0)=Pn 27 This test was proposed by Diewert (1992a, p. 216).

(1911, p. 405) proposed the related test P( p0, p1, q0, i=1p0 i =Pn i=1p1 i q0 i : i q0 294 equation (16.17), the corresponding price index P must satisfy T8.
This argument provides some additional justiﬁcation for assuming the validity of T8 for the price index function P. 16.41 T7 and T8 together impose the property that the price index P does not depend on the absolute magnitudes of the quantity vectors q0 and q1.

Invariance and symmetry tests 16.42 The next ﬁve tests, T9–T13, are invariance or symmetry tests.

Fisher (1922, pp.
Fisher (1922, pp.

62–63, 458–460) and Walsh (1901, p. 105; 1921b, p. 542) seem to have been the ﬁrst researchers to appreciate the signiﬁcance of these kinds of tests.

Fisher (1922, pp.

62–63) spoke of fairness but it is clear that he had symmetry proper- ties in mind.
It is perhaps unfortunate that he did not realize that there were more symmetry and invar- iance properties than the ones he proposed; if he had, it is likely that he would have been able to provide an axiomatic characterization for his ideal price index, as is done in paragraphs 16.53 to 16.56.
The ﬁrst invariance test is that the price index should remain unchanged if the ordering of the commodities is changed: T9: Commodity reversal test (or invariance to changes in the ordering of commodities): P( p0*, p1*, q0*, q1*)=P( p0, p1, q0, q1) where pt* denotes a permutation of the components of the vector pt and qt* denotes the same permutation of the components of qt for t=0,1.

This test is attribu- table to Fisher (1922, p. 63)28 and it is one of his three famous reversal tests.
The other two are the time reversal test and the factor reversal test, which are considered below.

16.43 The next test asks that the index be invariant to changes in the units of measurement.

T10: Invariance to changes in the units of measurement (commensurability test): 1, .

.

.

, anp1 1, .

.

.

, anp0 P(a1p0 1, .

.

.

, a1 1 q1 a1 n; q1 1, .

.

.

, q0 q0 n; a1p1 n q1 n)=P( p0 1, .

.

.

, q1 n) 1 q0 n; p1 1, .

.

.

, a1 n q0 n; 1, .

.

.

, p1 n; n; a1 1, .

.

.


, p0 for all a1 > 0, .

.

.

, an > 0 That is, the price index does not change if the units of measurement for each commodity are changed.

The concept of this test is attributable to Jevons (1863, p. 23) and the Dutch economist Pierson (1896, p. 131), who criticized several index number formulae for not satis- fying this fundamental test.
Fisher (1911, p. 411) ﬁrst called this test the change of units test; later, Fisher (1922, p. 420) called it the commensurability test.

28 ‘‘This [test] is so simple as never to have been formulated.

It is merely taken for granted and observed instinctively.

Any rule for averaging the commodities must be so general as to apply interchangeably to all of the terms averaged’’ (Fisher (1922, p. 63)).

16.44 The next test asks that the formula be invar- iant to the period chosen as the base period.
that T13 is equivalent to the following property for the associated quantity index Q: THE AXIOMATIC AND STOCHASTIC APPROACHES TO INDEX NUMBER THEORY T11: Time reversal test: P( p0, p1, q0, q1)=1=P( p1, p0, q1, q0) That is, if the data for periods 0 and 1 are interchanged, then the resulting price index should equal the reciprocal of the original price index.
Obviously, in the one good case when the price index is simply the single price ratio, this test will be satisﬁed (as are all the other tests listed in this section).

When the number of goods is greater than one, many commonly used price indices fail this test; e.g., the Laspeyres (1871) price index, PL deﬁned by equation (15.5) in Chapter 15, and the Paasche (1874) price index, PP deﬁned by equation (15.6) in Chapter 15, both fail this fundamental test.
The concept of the test is attributable to Pierson (1896, p. 128), who was so upset by the fact that many of the commonly used index number formulae did not satisfy this test that he pro- posed that the entire concept of an index number should be abandoned.

More formal statements of the test were made by Walsh (1901, p. 368; 1921b, p. 541) and Fisher (1911, p. 534; 1922, p. 64).
16.45 The next two tests are more controversial, since they are not necessarily consistent with the eco- nomic approach to index number theory.

These tests are, however, quite consistent with the weighted sto- chastic approach to index number theory, discussed later in this chapter.

T12: Quantity reversal test (quantity weights symmetry test): P( p0, p1, q0, q1)=P( p0, p1, q1, q0) That is, if the quantity vectors for the two periods are interchanged, then the price index remains invariant.
This property means that if quantities are used to weight the prices in the index number formula, then the period 0 quantities q0 and the period 1 quantities q1 must enter the formula in a symmetric or even-handed manner.

Funke and Voeller (1978, p. 3) introduced this test; they called it the weight property.
16.46 The next test is the analogue to T12 applied to quantity indices: T13: Price reversal test (price weights symmetry test):29 Q( p0, p1, q0, q1)=Q( p1, p0, q0, q1) (16:19) That is, if the price vectors for the two periods are interchanged, then the quantity index remains invari- ant.

Thus if prices for the same good in the two periods are used to weight quantities in the construction of the quantity index, then property T13 implies that these prices enter the quantity index in a symmetric manner.
Mean value tests 16.47 The next three tests, T14–T16, are mean value tests.

T14: Mean value test for prices:30 mini ( p1 i =p0 maxi ( p1 i : i=1, .

.

.

, n) P( p0, p1, q0, q1) i =p0 i : i=1, .

.

.

, n) (16:20) That is, the price index lies between the minimum price ratio and the maximum price ratio.

Since the price index is supposed to be interpreted as some sort of an average of the n price ratios, p1 i , it seems essential that the price index P satisfy this test.
i =p0 16.48 The next test is the analogue to T14 applied to quantity indices: T15: Mean value test for quantities:31 mini (q1 i =q0 i : i=1, .

.

.

, n) (V 1=V 0) P( p0, p1, q0, q1) maxi (q1 i =q0 i : i=1, .

.

.

, n) (16:21) where V t is the period t value for the aggregate deﬁned by equation (16.1).
Using the product test (16.17) to deﬁne the quantity index Q in terms of the price index P, it can be seen that T15 is equivalent to the following property for the associated quantity index Q: mini (q1 i =q0 maxi (q1 i : i=1, .

.

.

, n) Q( p0, p1, q0, q1) i =q0 i : i=1, .

.

.

, n) (16:22) , 0 B B @ n P i=1 n P i=1 i q1 p1 i i q0 p0 i 1 C C A P( p0, p1, q0, q1)= , 0 B B @ n P i=1 n P i=1 i q1 p0 i i q0 p1 i 1 C C A P( p1, p0, q0, q1) (16:18) i of the individual quantities.
That is, the implicit quantity index Q deﬁned by P lies between the minimum and maximum rates of growth i =q0 q1 16.49 In paragraphs 15.18 to 15.32 of Chapter 15, it was argued that it is very reasonable to take an average of the Laspeyres and Paasche price indices as a single ‘‘best’’ measure of overall price change.

This point of Thus if we use equation (16.17) to deﬁne the quantity index Q in terms of the price index P, then it can be seen 29 This test was proposed by Diewert (1992a, p. 218).
30 This test seems to have been ﬁrst proposed by Eichhorn and Voeller (1976, p. 10).

31 This test was proposed by Diewert (1992a, p. 219).

295 CONSUMER PRICE INDEX: THEORY AND PRACTICE view can be turned into a test: T20: Monotonicity in base quantities: if q0 < q2, then T16: Paasche and Laspeyres bounding test:32 The price index P lies between the Laspeyres and Paasche indices, PL and PP, deﬁned by equations (15.5) and (15.6) in Chapter 15.
A test could be proposed where the implicit quantity index Q that corresponds to P via equation (16.17) is to lie between the Laspeyres and Paasche quantity indices, QP and QL, deﬁned by equations (15.10) and (15.11) in Chapter 15.

However, the resulting test turns out to be equivalent to test T16.
Monotonicity tests 16.50 The ﬁnal four tests, T17–T20, are mono- tonicity tests; index P( p0, p1, q0, q1) change as any component of the two price vectors p0 and p1 increases or as any component of the two quantity vectors q0 and q1 increases?

i.e., how should the price T17: Monotonicity in current prices: P( p0, p1, q0, q1) < P( p0, p2, q0, q1) if p1 < p2 That is, if some period 1 price increases, then the price index must increase, so that P( p0, p1, q0, q1) is increasing in the components of p1.
This property was proposed by Eichhorn and Voeller (1976, p. 23) and it is a very reasonable property for a price index to satisfy.

T18: Monotonicity in base prices: P( p0, p1, q0, q1) > P( p2, p1, q0, q1) if p0 < p2 That is, if any period 0 price increases, then the price index must decrease, so that P( p0, p1, q0, q1) is decreas- ing in the components of p0.

This very reasonable property was also proposed by Eichhorn and Voeller (1976, p. 23).
T19: Monotonicity in current quantities: if q1 < q2, then 0 B B @ n P i=1 n P i=1 1 C C A i q1 p1 i i q0 p0 i P( p0, p1, q0, q1) > 0 B B @ n P i=1 n P i=1 1 C C A i q1 p1 i p0 i q2 i P( p0, p1, q2, q1) (16:24) 16.51 Let Q be the implicit quantity index that corresponds to P using equation (16.17).
Then it is found that T19 translates into the following inequality involving Q: Q( p0, p1, q0, q1) < Q( p0, p1, q0, q2) if q1 < q2 (16:25) That is, if any period 1 quantity increases, then the implicit quantity index Q that corresponds to the price index P must increase.

Similarly, we ﬁnd that T20 translates into: Q( p0, p1, q0, q1) > Q( p0, p1, q2, q1) if q0 < q2 (16:26) That is, if any period 0 quantity increases, then the implicit quantity index Q must decrease.
Tests T19 and T20 are attributable to Vogt (1980, p. 70).

16.52 This concludes the listing of tests.

The next section oﬀers an answer to the question of whether any index number formula P( p0, p1, q0, q1) exists that can satisfy all 20 tests.
The Fisher ideal index and the test approach 16.53 It can be shown that the only index number formula P( p0, p1, q0, q1) which satisﬁes tests T1–T20 is the Fisher ideal price index PF deﬁned as the geometric mean of the Laspeyres and Paasche indices:33 PF ( p0, p1, q0, q1) fPL( p0, p1, q0, q1)Pp( p0, p1, q0, q1)g1=2 (16:27) 0 B B @ n P i=1 n P i=1 i q1 p1 i i q0 p0 i < 0 B B @ n P i=1 n P i=1 i q2 p1 i i q0 p0 i 1 C C A P(p0,p1,q0,q1) 1 C C A P(p0,p1,q0,q2) 16.54 It is relatively straightforward to show that the Fisher index satisﬁes all 20 tests.
The more diﬃcult part of the proof is to show that the Fisher index is the only index number formula that satisﬁes these tests.

This part of the proof follows from the fact that, if P satisﬁes the positivity test T1 and the three reversal tests, T11–T13, then P must equal PF.

To see this, rear- range the terms in the statement of test T13 into the (16:23) 32 Bowley (1901, p. 227) and Fisher (1922, p. 403) both endorsed this property for a price index.

33 See Diewert (1992a, p. 221).


296 THE AXIOMATIC AND STOCHASTIC APPROACHES TO INDEX NUMBER THEORY following equation: = P( p0, p1, q0, q1) P( p1, p0, q0, q1) n P i=1 n P i=1 i q1 p1 i p0 i q1 i i q0 p0 i n P i=1 n P i=1 P( p0, p1, q0, q1) P( p1, p0, q1, q0) i q0 p1 i = using T12, the quantity reversal test =P( p0, p1, q0, q1)P( p0, p1, q0, q1) using T11, the time reversal test (16:28) Now take positive square roots of both sides of equation the left-hand side of (16.28).
It can be seen that index PF ( p0, p1, q0, q1) the equation is the Fisher deﬁned by equation (16.27) and the right-hand side is P( p0, p1, q0, q1).

Thus if P satisﬁes T1, T11, T12 and T13, it must equal the Fisher ideal index PF.

16.55 The quantity index that corresponds to the Fisher price index using the product test (16.17) is QF, the Fisher quantity index, deﬁned by equation (15.14) in Chapter 15.

16.56 It turns out that PF satisﬁes yet another test, T21, which was Fisher’s (1921, p. 534; 1922, pp.
72–81) third reversal test (the other two being T9 and T11): T21: Factor reversal test (functional form symmetry test): P( p0, p1, q0, q1)P(q0, q1, p0, p1)= n P i=1 n P i=1 i q1 p1 i i q0 p0 i (16:29) A justiﬁcation for this test is the following: if P( p0, p1, q0, q1) is a good functional form for the price index, then, if the roles of prices and quantities are reversed, P(q0, q1, p0, p1) ought to be a good functional form for a quantity index (which seems to be a correct argument) and thus the product of the price index P( p0, p1, q0, q1) and the quantity index Q( p0, p1, q0, q1)= P(q0, q1, p0, p1) ought to equal the value ratio, V1/V 0 .
The second part of this argument does not seem to be valid, and thus many researchers over the years have objected to the factor reversal test.

Nevertheless, if T21 is accepted as a basic test, Funke and Voeller (1978, p. 180) showed that the only index number function P( p0, p1, q0, q1) which satisﬁes T1 (positivity), T11 (time reversal test), T12 (quantity reversal test) and T21 (factor reversal test) is the Fisher ideal index PF deﬁned by equation (16.27).
Thus the price reversal test T13 can be replaced by the factor reversal test in order to obtain a minimal set of four tests that lead to the Fisher price index.34 The test performance of other indices 16.57 The Fisher price index PF satisﬁes all 20 of the tests T1–T20 listed above.

Which tests do other com- 34 Other characterizations of the Fisher price index can be found in Funke and Voeller (1978) and Balk (1985; 1995).

monly used price indices satisfy?


Recall the Laspeyres index PL deﬁned by equation (15.5), the Paasche index PP deﬁned by equation (15.6), the Walsh index PW deﬁned by equation (15.19) and the To¨ rnqvist index PT deﬁned by equation (15.81) in Chapter 15.

16.58 Straightforward computations show that the Paasche and Laspeyres price indices, PL and PP, fail only the three reversal tests, T11, T12 and T13.
Since the quantity and price reversal tests, T12 and T13, are somewhat controversial and hence can be discounted, the test performance of PL and PP seems at ﬁrst sight to be quite good.

The failure of the time reversal test, T11, is nevertheless a severe limitation associated with the use of these indices.
16.59 The Walsh price index, PW, fails four tests: T13, the price reversal test; T16, the Paasche and Las- peyres bounding test; T19, the monotonicity in current quantities test; and T20, the monotonicity in base quan- tities test.

16.60 Finally, the To¨ rnqvist price index PT fails nine tests: T4 (the ﬁxed basket test), the quantity and price reversal tests T12 and T13, T15 (the mean value test for quantities), T16 (the Paasche and Laspeyres bounding test) and the four monotonicity tests T17 to T20.
Thus the To¨ rnqvist index is subject to a rather high failure rate from the viewpoint of this axiomatic approach to index number theory.35 16.61 The tentative conclusion that can be drawn from the above results is that, from the viewpoint of this particular bilateral test approach to index num- bers, the Fisher ideal price index PF appears to be ‘‘best’’ since it satisﬁes all 20 tests.

The Paasche and Laspeyres indices are next best if we treat each test as being equally important.
Both of these indices, however, fail the very important time reversal test.

The remaining two indices, the Walsh and To¨ rnqvist price indices, both satisfy the time reversal test but the Walsh index emerges as being ‘‘better’’ since it passes 16 of the 20 tests whereas the To¨ rnqvist only satisﬁes 11 tests.36 The additivity test test 16.62 There is an additional that many national income accountants regard as very impor- tant: the additivity test.
This is a test or property implicit quantity index that Q( p0, p1, q0, q1) that corresponds to the price index P( p0, p1, q0, q1) using the product (16.17).

This test states that the implicit quantity index has the is placed on the test 35 It is shown in Chapter 19, however, that the To¨ rnqvist index approximates the Fisher index quite closely using ‘‘normal’’ time series data that are subject to relatively smooth trends.
Hence, under these circumstances, the To¨ rnqvist index can be regarded as passing the 20 tests to a reasonably high degree of approximation.

36 This assertion needs to be qualiﬁed: there are many other tests that we have not discussed, and price statisticians might hold diﬀerent opinions regarding the importance of satisfying various sets of tests.

Other tests are discussed by von Auer (2001; 2002), Eichhorn and Voeller (1976), Balk (1995) and Vogt and Barta (1997), among others.
It is shown in paragraphs 16.101 to 16.135 that the To¨ rnqvist index is ideal when considered under a diﬀerent set of axioms.

297 CONSUMER PRICE INDEX: THEORY AND PRACTICE following form: Q( p0, p1, q0, q1)= n P i=1 n P m=1 i q1 p* i mq0 p* m (16:30) 16.65 As was the case with the theory of the unequiv- ocal price index, it can be seen that if the unequivocal quantity index QK is to satisfy the time reversal test (16.32), the mean function in equation (16.31) must be symmetric.
It is also asked that QK satisfy the follow- ing invariance to proportional changes in current prices test.

where the common across-periods price for commodity i, p* i for i=1, .

.

.

, n, can be a function of all 4n prices and quantities pertaining to the two periods or situations under consideration, p0, p1, q0, q1.
In the literature on making multilateral comparisons (i.e., comparisons between more than two situations), it is quite common to assume that the quantity comparison between any two regions can be made using the two regional quan- tity vectors, q0 and q1, and a common reference price vector, P* ( p* 1, .

.

.

, p* 16.63 Obviously, diﬀerent versions of the additivity test can be obtained if further restrictions are placed on precisely which variables each reference price p* i depends.
The simplest such restriction is to assume that each p* i depends only on the commodity i prices per- taining to each of the two situations under consideration, i and p1 p0 i .

If it is further assumed that the functional form for the weighting function is the same for each commodity, so that p* i ) for i=1, .

.

.

, n, then we are led to the unequivocal quantity index postulated by Knibbs (1924, p. 44).
i =m( p0 i , p1 n):37 16.64 The theory of the unequivocal quantity index (or the pure quantity index)38 parallels the theory of the pure price index outlined in paragraphs 15.24 to 15.32 of Chapter 15.

An outline of this theory is given here.


Let the pure quantity index QK have the following func- tional form: i , p1 i ) k, p1 k) (16:31) i m( p0 q1 km( p0 q0 QK ( p0, p1, q0, q1) n P i=1 n P k=1 It is assumed that the price vectors p0 and p1 are strictly positive and the quantity vectors q0 and q1 are non-nega- tive but have at least one positive component.39 The pro- blem is to determine the functional form for the averaging function m if possible.
To do this, it is necessary to impose some tests or properties on the pure quantity index QK.

As was the case with the pure price index, it is very reasonable to ask that the quantity index satisfy the time reversal test: QK ( p1, p0, q1, q0)= 1 QK ( p0, p1, q0, q1) (16:32) 37 Hill (1993, p. 395–397) termed such multilateral methods the block approach while Diewert (1996a, pp.

250–251) used the term average price approaches.

Diewert (1999b, p. 19) used the term additive multi- lateral system.
For axiomatic approaches to multilateral index number theory, see Balk (1996a; 2001) and Diewert (1999b).

38 Diewert (2001) used this term.

39 It is assumed that m(a, b) has the following two properties: m(a, b) is a positive and continuous function, deﬁned for all positive numbers a and b, and m(a, a)=a for all a>0.
298 QK ( p0, lp1, q0, q1)=QK ( p0, p1, q0, q1) for all p0, p1, q0, q1and all l > 0 (16:33) 16.66 The idea behind this invariance test is this: the quantity index QK ( p0, p1, q0, q1) should depend only on the relative prices in each period and it should not depend on the amount of inﬂation between the two periods.

Another way to interpret test (16.33) is to look at what the test implies for the corresponding implicit price index, PIK, deﬁned using the product test (16.17).
It can be shown that if QK satisﬁes equation (16.33), then the corresponding implicit price index PIK will satisfy test T5 above, the proportionality in current prices test.
The two tests, (16.32) and (16.33), determine the precise functional form for the pure quantity index QK deﬁned by equation (16.31): the pure quantity index or Knibbs’ unequivocal quantity index QK must be the Walsh quantity index QW 40 deﬁned by: QW ( p0, p1, q0, q1) n P i=1 n P k=1 p ﬃﬃﬃﬃﬃﬃﬃﬃﬃ p0 i p1 i q1 i q ﬃﬃﬃﬃﬃﬃﬃﬃﬃ p0 kp1 k q0 k (16:34) 16.67 Thus with the addition of two tests, the pure price index PK must be the Walsh price index PW deﬁned by equation (15.19) in Chapter 15 and with the addition of the same two tests (but applied to quantity indices instead of price indices), the pure quantity index QK must be the Walsh quantity index QW deﬁned by equation (16.34).
Note, however, that the product of the Walsh price and quantity indices is not equal to the expenditure ratio, V1/V 0.
Thus believers in the pure or unequivocal price and quantity index concepts have to choose one of these two concepts; they cannot both apply simultaneously.41 16.68 If the quantity index Q( p0, p1, q0, q1) satis- ﬁes the additivity test (16.30) for some price weights p* i , then the percentage change in the quantity aggregate, Q( p0, p1, q0, q1)1, can be rewritten as follows: Q( p0, p1, q0, q1)1= n P i=1 n P m=1 n =P i=1 n P i=1 1= i q1 p* i mq0 p* m wi(q1 i q0 i ) n i q1 p* i P mq0 p* m m=1 n P m=1 mq0 p* m (16:35) 40 This is the quantity index that corresponds to the price index 8 deﬁned by Walsh (1921a, p. 101).
41 Knibbs (1924) did not notice this point.

where the weight for commodity i, wi, is deﬁned as obtained for the Fisher ideal quantity index: THE AXIOMATIC AND STOCHASTIC APPROACHES TO INDEX NUMBER THEORY wi p* i n P m=1 mq0 p* m ; i=1, .

.

.

, n (16:36) Note that the change in commodity i going from situ- ation 0 to situation 1 is q1 i q0 i .
Thus the ith term on the right-hand side of equation (16.35) is the contribution of the change in commodity i to the overall percentage change in the aggregate going from period 0 to 1.

Busi- ness analysts often want statistical agencies to provide decompositions such as equation (16.35) so that they can decompose the overall change in an aggregate into sector- speciﬁc components of change.42 Thus there is a demand on the part of users for additive quantity indices.
16.69 For the Walsh quantity index deﬁned by equation (16.34), the ith weight is p ﬃﬃﬃﬃﬃﬃﬃﬃﬃ p0 i p1 i p ﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃ mp1 p0 m q0 m wWi n P m=1 ; i=1, .

.

.

, n (16:37) Thus the Walsh quantity index QW has a percentage decomposition into component changes of the form of equation (16.35), where the weights are deﬁned by equation (16.37).
16.70 It turns out that the Fisher quantity index QF, deﬁned by equation (15.14) in Chapter 15, also has an additive percentage change decomposition of the form given by equation (16.35).43 The ith weight wFi for this Fisher decomposition is rather complicated and depends on the Fisher quantity index QF( p0, p1, q0, q1) as follows:44 wFi i +(QF )2w1 w0 1+QF i ; i=1, .

.

.
.

.

, n (16:38) where QF is the value of the Fisher quantity index, QF ( p0, p1, q0, q1), and the period t normalized price for commodity i, wt is deﬁned as the period i price pt i, i divided by the period t expenditure on the aggregate: wt i pt i n P m=1 mqt pt m ; t=0, 1; i=1, .

.

.
.

.

, n (16:39) 16.71 Using the weights wFi deﬁned by equations (16.38) and (16.39), the following exact decomposition is 42 Business and government analysts also often demand an analogous decomposition of the change in price aggregate into sector-speciﬁc components that add up.

43 The Fisher quantity index also has an additive decomposition of the type deﬁned by equation (16.30) attributable to Van Ijzeren (1987, p. 6).

The ith reference price p* i +(1=2)p1 i = i PF ( p0, p1, q0, q1) for i=1, .

.

.
.

.

, n and where PF is the Fisher price index.

This decomposition was also independently derived by Dikhanov (1997).

The Van Ijzeren decomposition for the Fisher quantity index is currently being used by the US Bureau of Economic Analysis; see Moulton and Seskin (1999, p. 16) and Ehemann, Katz and Moulton (2002).

44 This decomposition was obtained by Diewert (2002a) and Reinsdorf, Diewert and Ehemann (2002).

For an economic interpretation of this decomposition, see Diewert (2002a).
is deﬁned as p* i (1=2)p0 QF ( p0, p1, q0, q1)1=P n i=1 wFi (q1 i q0 i ) (16:40) Thus the Fisher quantity index has an additive percen- tage change decomposition.45 16.72 Because of the symmetric nature of the Fisher price and quantity indices, it can be seen that the Fisher price index PF deﬁned by equation (16.27) also has the following additive percentage change decomposition: PF ðp0, p1, q0, q1Þ1=P n i=1 vFi ( p1 i p0 i ) (16:41) where the commodity i weight vFi is deﬁned as vFi i +(PF )2v1 v0 1+PF i ; i=1, .
.

.

, n (16:42) where PF is the value of the Fisher price index, PF( p0, p1, q0, q1), and the period t normalized quantity for commodity i, vt i, is deﬁned as the period i quantity qt i divided by the period t expenditure on the aggregate: vt i qt i n P m=1 mqt pt m ; t=0, 1; i=1, .

.

.
.

.

, n (16:43) 16.73 The above results show that the Fisher price and quantity indices have exact additive decompositions into components that give the contribution to the overall change in the price (or quantity) index of the change in each price (or quantity).
The stochastic approach to price indices The early unweighted stochastic approach 16.74 The stochastic approach to the determination of the price index can be traced back to the work of Jevons (1863; 1865) and Edgeworth (1888) over 100 years ago.46 The basic idea behind the (unweighted) stochastic approach is that each price relative, p1 i for i=1, 2, .

.

.

, n, can be regarded as an estimate of a com- mon inﬂation rate a between periods 0 and 1.47 i =p0 It is assumed that p1 i p0 i =a+ei; i=1, 2, .

.
.

.

, n (16:44) where a is the common inﬂation rate and the ei are random variables with mean 0 and variance s2.

The least 45 To verify the exactness of the decomposition, substitute equation (16.38) into equation (16.40) and solve the resulting equation for QF.

It is found that the solution is equal to QF deﬁned by equation (15.14) in Chapter 15.

46 For references to the literature, see Diewert (1993a, pp.

37–38; 1995a; 1995b).


47 ‘‘In drawing our averages the independent ﬂuctuations will more or less destroy each other; the one required variation of gold will remain undiminished’’ (Jevons (1863, p. 26)).
299 CONSUMER PRICE INDEX: THEORY AND PRACTICE squares or maximum likelihood estimator for a is the Carli (1764) price index PC deﬁned as cism towards this approach, which was vigorously advocated by Edgeworth (1923): PC( p0, p1) P n i=1 1 n p1 i p0 i (16:45) A drawback of the Carli price index is that it does i.e., PC( p1, p0)=1/ not satisfy the time reversal test, PC ( p0, p1).48 16.75 Now change the stochastic speciﬁcation and the logarithm of each price relative, assume that ln( p1 i =p0 i ), is an unbiased estimate of the logarithm of the inﬂation rate between periods 0 and 1, b say.
The counterpart to equation (16.44) is: ln p1 i p0 i =b+ei; i=1, 2, .

.

.

, n (16:46) where b:ln a and the ei are independently distributed random variables with mean 0 and variance s2.

The least squares or maximum likelihood estimator for b is the logarithm of the geometric mean of the price relatives.
Hence the corresponding estimate for the common inﬂation rate a49 is the Jevons (1865) price index PJ deﬁned as follows: PJ( p0, p1) s n n Y i=1 ﬃﬃﬃﬃﬃ p1 i p0 i : (16:47) 16.76 The Jevons price index PJ does satisfy the time reversal test and hence is much more satisfactory than the Carli index PC.
Both the Jevons and Carli price indices nevertheless suﬀer from a fatal ﬂaw: each price relative p1 i is regarded as being equally important and is given an equal weight in the index number formulae (16.45) and (16.47).
John Maynard Keynes was parti- cularly critical of this unweighted stochastic approach to index number theory.50 He directed the following criti- i =p0 i =p0 i =p0 i=1 ln( p1 48 In fact, Fisher (1922, p. 66) noted that PC( p0, p1)PC ( p1, p0) 1 unless the period 1 price vector p1 is proportional to the period 0 price vector p0; i.e., Fisher showed that the Carli index has a deﬁnite upward bias.

He urged statistical agencies not to use this formula.

Walsh (1901, pp.
Walsh (1901, pp.

331, 530) also discovered this result for the case n=2.

49 Greenlees (1999) pointed out that although (1/n) Pn i ) is an unbiased estimator for b, the corresponding exponential of this estimator, PJ deﬁned by equation (16.47), will generally not be an unbiased estimator for a under our stochastic assumptions.

To see this, let xi= ln p1 i .

Taking expectations, we have: Exi=b= ln a. Deﬁne the positive, convex function f of one variable x by f(x):ex.
By Jen- sen’s (1906) inequality, Ef(x) f(Ex).

Letting x equal the random i =p0 i )=Ef (xi) f (Exi)= variable xi, f (b)=eb=eln a=a: Thus for each n, E( p1 i ) a, and it can be seen that the Jevons price index will generally have an upward bias under the usual stochastic assumptions.
50 Walsh (1901, p. 83) also stressed the importance of proper weighting according to the economic importance of the commodities in the to assign uneven weighting with periods being compared: approximation to the relative sizes, either over a long series of years or for every period separately, would not require much additional trou- ble; and even a rough procedure of this sort would yield results far superior to those yielded by even weighting.
It is especially absurd to refrain from using roughly reckoned uneven weighting on the ground that it is not accurate, and instead to use even weighting, which is much more inaccurate.’’ this inequality becomes: E( p1 i =p0 ‘‘But 300 Nevertheless I venture to maintain that such ideas, which I have endeavoured to expound above as fairly and as plausibly as I can, are root-and-branch erroneous.
The ‘‘errors of observation’’, the ‘‘faulty shots aimed at a single bull’s eye’’ conception of the index number of prices, Edgeworth’s ‘‘objective mean variation of general prices’’, is the result of confusion of thought.

There is no bull’s eye.

There is no moving but unique centre, to be called the general price level or the objective mean variation of general prices, round which are scattered the moving price levels of individual things.
There are all the various, quite deﬁnite, conceptions of price levels of composite com- modities appropriate for various purposes and inquiries which have been scheduled above, and many others too.

There is nothing else.

Jevons was pursuing a mirage.

What is the ﬂaw in the argument?

In the ﬁrst place it assumed that the ﬂuctuations of individual prices round the ‘‘mean’’ are ‘‘random’’ in the sense required by the theory of the combination of independent observations.
In this theory the divergence of one ‘‘observation’’ from the true position is assumed to have no inﬂuence on the divergences of other ‘‘observations’’.
But in the case of prices, a movement in the price of one commodity necessarily inﬂuences the movement in the prices of other commodities, whilst the magnitudes of these compensa- tory movements depend on the magnitude of the change in expenditure on the ﬁrst commodity as compared with the importance of the expenditure on the commodities secondarily aﬀected.
Thus, instead of ‘‘independence’’, there is between the ‘‘errors’’ in the successive ‘‘observa- tions’’ what some writers on probability have called ‘‘connexity’’, or, as Lexis expressed it, there is ‘‘sub-nor- mal dispersion’’.

We cannot, therefore, proceed further until we have enunciated the appropriate law of connexity.
But the law of connexity cannot be enunciated without reference to the relative importance of the commodities aﬀected— which brings us back to the problem that we have been trying to avoid, of weighting the items of a composite commodity (Keynes (1930, pp.

76–77)).

The main point Keynes seemed to be making in the above quotation is that prices in the economy are not independently distributed from each other and from quantities.
In current macroeconomic terminology, Keynes can be interpreted as saying that a macro- economic shock will be distributed across all prices and quantities in the economy through the normal inter- action between supply and demand; i.e., through the workings of the general equilibrium system.

Thus Keynes seemed to be leaning towards the economic approach to index number theory (even before it was developed to any great extent), where quantity move- ments are functionally related to price movements.
A second point that Keynes made in the above quotation is that there is no such thing as the inﬂation rate; there are only price changes that pertain to well-speciﬁed sets i.e., the domain of of commodities or transactions; deﬁnition of the price index must be carefully speci- ﬁed.51 A ﬁnal point that Keynes made is that price 51 See paragraphs 15.7 to 15.17 in Chapter 15 for additional discussion on this point.
THE AXIOMATIC AND STOCHASTIC APPROACHES TO INDEX NUMBER THEORY movements must be weighted by their economic importance, i.e., by quantities or expenditures.
16.77 In addition to the above theoretical criticisms, Keynes also made the following strong empirical attack on Edgeworth’s unweighted stochastic approach: The Jevons–Edgeworth ‘‘objective mean variation of general prices’’, or ‘‘indeﬁnite’’ standard, has generally been identiﬁed, by those who were not as alive as Edge- worth himself was to the subtleties of the case, with the purchasing power of money – if only for the excellent reason that it was diﬃcult to visualise it as anything else.
And since any respectable index number, however weighted, which covered a fairly large number of com- modities could, in accordance with the argument, be regarded as a fair approximation to the indeﬁnite stan- dard, it seemed natural to regard any such index as a fair approximation to the purchasing power of money also.
Finally, the conclusion that all the standards ‘‘come to much the same thing in the end’’ has been reinforced ‘‘inductively’’ by the fact that rival index numbers (all of them, however, of the wholesale type) have shown a considerable measure of agreement with one another in spite of their diﬀerent compositions .

.

.

On the contrary, the tables given above (pp.


53, 55) supply strong pre- sumptive evidence that over long period as well as over short period the movements of the wholesale and of the consumption standards respectively are capable of being widely divergent (Keynes (1930, pp.

80–81)).
80–81)).

In the above quotation, Keynes noted that the propo- nents of the unweighted stochastic approach to price change measurement were comforted by the fact that all of the then existing (unweighted) indices of wholesale prices showed broadly similar movements.

Keynes showed empirically, however, that his wholesale price indices moved quite diﬀerently from his consumer price indices.
In order to overcome the above criticisms of the unweighted stochastic approach to index numbers, it is necessary to: have a deﬁnite domain of deﬁnition for the index 16.78 number; weight the price relatives by their economic impor- tance.52 Alternative methods of weighting are discussed in the following sections.

The weighted stochastic approach 16.79 Walsh (1901, pp.
88–89) seems to have been the ﬁrst index number theorist to point out that a sen- sible stochastic approach to measuring price change means that individual price relatives should be weighted according to their economic importance or their trans- actions value in the two periods under consideration: It might seem at ﬁrst sight as if simply every price quotation were a single item, and since every commodity (any kind of commodity) has one price-quotation attached to it, it would seem as if price-variations of every kind of commodity were the single item in question.
This is the way the question struck the ﬁrst inquirers into 52 Walsh (1901, pp.

82–90; 1921a, pp.

82–83) also objected to the lack of weighting in the unweighted stochastic approach to index number theory.

price-variations, wherefore they used simple averaging with even weighting.

But a price-quotation is the quota- tion of the price of a generic name for many articles; and one such generic name covers a few articles, and another covers many.

.

.

.
.

.

.

A single price-quotation, therefore, may be the quotation of the price of a hundred, a thousand, or a million dollar’s worths, of the articles that make up the commodity named.

Its weight in the averaging, therefore, ought to be according to these money-unit’s worth (Walsh (1921a, pp.

82–83)).

But Walsh did not give a convincing argument on exactly how these economic weights should be determined.

i q0 price kq0 k, change k=1p0 i =Pn i p0 is Pn logarithmic 16.80 Henri Theil (1967, pp.
136–137) proposed a solution to the lack of weighting in the Jevons index, PJ deﬁned by equation (16.47).

He argued as follows.

Suppose we draw price relatives at random in such a way that each dollar of expenditure in the base period has an equal chance of being selected.

Then the prob- ability that we will draw the ith price relative is equal to s0 the period 0 expenditure share for commodity i.
Then the overall mean (period 0 i=1s0 weighted) i i ):53 Now repeat the above mental experiment ln ( p1 i =p0 and draw price relatives at random in such a way that each dollar of expenditure in period 1 has an equal probability of being selected.
This leads to the overall mean (period 1 weighted) logarithmic price change of Pn i=1s1 i 16.81 Each of these measures of overall logarithmic price change seems equally valid, so we could argue for taking a symmetric average of the two measures in order to obtain a ﬁnal single measure of overall logarithmic price change.
Theil55 argued that a ‘‘nice’’ symmetric index number formula can be obtained if the probability of selection for the nth price relative is made equal to the arithmetic average of the period 0 and 1 expenditure shares for commodity n. Using these probabilities of selection, Theil’s ﬁnal measure of overall logarithmic price change was ln ( p1 i ):54 i =p0 ln PT ( p0, p1, q0, q1) P n i=1 1 2 (s0 i +s1 i ) ln p1 i p0 i (16:48) Note that the index PT deﬁned by equation (16.48) is equal to the To¨ rnqvist index deﬁned by equation (15.81) in Chapter 15.
16.82 A statistical interpretation of the right-hand side of equation (16.48) can be given.

Deﬁne the ith 53 In Chapter 19, this index is called the geometric Laspeyres index, PGL.

Vartia (1978, p. 272) referred to this index as the logarithmic Laspeyres index.

Yet another name for the index is the base weighted geometric index.

54 In Chapter 19, this index is called the geometric Paasche index, PGP.

Vartia (1978, p. 272) referred to this index as the logarithmic Paasche index.
Yet another name for the index is the current period weighted geometric index.

55 ‘‘The price index number deﬁned in (1.8) and (1.9) uses the n individual logarithmic price diﬀerences as the basic ingredients.

They are combined linearly by means of a two-stage random selection procedure: First, we give each region the same chance ½ of being selected, and second, we give each dollar spent in the selected region the same chance (1/ma or 1/mb) of being drawn’’ (Theil (1967, p. 138)).
301 CONSUMER PRICE INDEX: THEORY AND PRACTICE logarithmic price ratio ri by: ri ln p1 i p0 i for i=1, .

.

.

, n (16:49) Now deﬁne the discrete random variable, R say, as the random variable which can take on the values ri with probabilities ri (1=2)[s0 i +s1 i=1, .

.

.

, n. Note i ] that, since each set of expenditure shares, s0 i and s1 i , sums to one over i, the probabilities ri will also sum to one.
It can be seen that the expected value of the discrete ran- dom variable R is for E R½ n P i=1 n riri=P i=1 1 2 (s0 i +s1 i ) ln p1 i p0 i = ln PT ( p0, p1, q0, q1): (16:50) i ] for i=1, .

.

.
, n. Thus the logarithm of the index PT can be interpreted as the expected value of the distribution of the logarithmic price ratios in the domain of deﬁnition under con- sideration, where the n discrete price ratios in this domain of deﬁnition are weighted according to Theil’s prob- ability weights, ri (1=2)[s0 i +s1 16.83 Taking antilogs of both sides of equation (16.48), the To¨ rnqvist (1936; 1937) and Theil price index, PT, is obtained.56 This index number formula has a number of good properties.
In particular, PT satisﬁes the proportionality in current prices test T5 and the time reversal test T11, discussed above.

These two tests can be used to justify Theil’s (arithmetic) method of forming an average of the two sets of expenditure shares in order to obtain his probability weights, ri (1=2)[s0 i +s1 i ] for i=1, .

.

.
.

.

, n. Consider the following symmetric mean class of logarithmic index number formulae: ln PS( p0, p1, q0, q1) P n i=1 m(s0 i , s1 i ) ln p1 i p0 i (16:51) i , s1 where m(s0 i ) is a positive function of the period 0 and 1 expenditure shares on commodity i, s0 i respec- tively.

In order for PS to satisfy the time reversal test, it is necessary that the function m be symmetric.

Then it can be shown57 that for PS to satisfy test T5, m must be the arithmetic mean.
This provides a reasonably strong jus- tiﬁcation for Theil’s choice of the mean function.

i and s1 16.84 The stochastic approach of Theil has another ‘‘nice’’ symmetry property.

Instead of considering the distribution of the price ratios ri= ln p1 i , we could also consider the distribution of the reciprocals of these price ratios, say: i =p0 ti ln p0 i p1 i = ln 1 p1 i p0 i = ln p1 i p0 i =ri for i=1, .

.

.
.

.

, n (16:52) 56 The sampling bias problem studied by Greenlees (1999) does not occur in the present context because there is no sampling involved in deﬁnition (16.50): the sum of the pt i over i for each period t is assumed to equal the value aggregate Vt for period t. 57 See Diewert (2000) and Balk and Diewert (2001).

i qt 302 i +s1 The symmetric probability, ri (1=2)[s0 i ], can still be associated with the ith reciprocal logarithmic price ratio ti for i=1, .

.

.
.

.

, n. Now deﬁne the discrete random variable, T say, as the random variable which can take on the values ti with probabilities ri (1=2)[s0 i ] for i=1, .

.

.
, n. It can be seen that the expected value of the discrete random variable T is i +s1 E T½ riti n P i=1 n = P i=1 =E R½ using (16:50) = ln PT ( p0, p1, q0, q1) riti using (16:52) (16:53) Thus it can be seen that the distribution of the random variable T is equal to minus the distribution of the random variable R. Hence it does not matter whether the distribution of the original logarithmic price ratios, ri ln p1 i =p0 i , is considered or the distribution of their reciprocals, ti ln p0 i , is considered: essentially the same stochastic theory is obtained.
i =p1 16.85 i =p0 It is possible to consider weighted stochastic approaches to index number theory where the dis- tribution of the price ratios, p1 i , is considered rather than the distribution of the logarithmic price ratios, ln p1 i =p0 i .

Thus, again following in the footsteps of Theil, suppose that price relatives are drawn at random in such a way that each dollar of expenditure in the base period has an equal chance of being selected.
Then the probability that the ith price relative will be drawn is equal to s0 i , the period 0 expenditure share for com- modity i.

Thus the overall mean (period 0 weighted) price change is: PL( p0, p1, q0, q1)=P n i=1 s0 i p1 i p0 i (16:54) which turns out to be the Laspeyres price index, PL.

This stochastic approach is the natural one for studying sam- pling problems associated with implementing a Laspeyres price index.
16.86 Now repeat the above mental experiment and draw price relatives at random in such a way that each dollar of expenditure in period 1 has an equal prob- ability of being selected.

This leads to the overall mean (period 1 weighted) price change equal to: PPAL( p0, p1, q0, q1)= P n i=1 s1 i p1 i p0 i (16:55) This is known as the Palgrave (1886) index number formula.58 16.87 It can be veriﬁed that neither the Laspeyres nor Palgrave price indices satisfy the time reversal test, T11.
Thus, again following in the footsteps of Theil, it might be attempted to obtain a formula that satisﬁes the time reversal test by taking a symmetric average of the 58 It is formula number 9 in Fisher’s (1922, p. 466) listing of index number formulae.

THE AXIOMATIC AND STOCHASTIC APPROACHES TO INDEX NUMBER THEORY two sets of shares.
Thus consider the following class of symmetric mean index number formulae: Pm( p0, p1, q0, q1) P n i=1 m(s0 i , s1 i ) p1 i p0 i (16:56) i , s1 where m(s0 i ) is a symmetric function of the period 0 and 1 expenditure shares for commodity i, s0 i and s1 i respectively.
In order to interpret the right hand-side of equation (16.56) as an expected value of the price ratios i =p0 p1 i , it is necessary that n P i=1 m(s0 i , s1 i )=1 (16:57) In order to satisfy equation (16.57), however, m must be the arithmetic mean.59 With this choice of m, equation (16.56) becomes the following (unnamed) index number formula, Pu: Pu( p0, p1, q0, q1) P n i=1 1 2 (s0 i +s1 i ) p1 i p0 i (16:58) Unfortunately, the unnamed index Pu does not satisfy the time reversal test either.60 16.88 i =p0 Instead of considering the distribution of the price ratios, p1 i , the distribution of the reciprocals of these price ratios could be considered.
The counterparts to the asymmetric indices deﬁned earlier by equations (16.54) and (16.55) are now Pn i=1s1 i ( p0 i ), respectively.

These are (stochastic) price indices going backwards from period 1 to 0.
In order to make these indices comparable with other previous forward- looking indices, take the reciprocals of these indices (which leads to harmonic averages) and the following two indices are obtained: i ) and Pn i=1s0 i ( p0 i =p1 i =p1 PHL( p0, p1, q0, q1) 1 s0 i p0 i p1 i n P i=1 (16:59) PHP( p0, p1, q0, q1) 1 = s1 i n P i=1 p0 i p1 i =PP( p0, p1, q0, q1) n P i=1 s1 i 1 1 p1 i p0 i (16:60) using equation (15.9) in Chapter 15.
Thus the reciprocal stochastic price index deﬁned by equation (16.60) turns out to equal the ﬁxed basket Paasche price index, PP.

This stochastic approach is the natural one for studying sampling problems associated with implementing a Paasche price index.

The other asymmetrically weighted 59 For a proof of this assertion, see Balk and Diewert (2001).

60 In fact, this index suﬀers from the same upward bias as the Carli index in that Pu( p0, p1, q0, q1)Pu( p1, p0, q1, q0)1.
To prove this, note that the previous inequality is equivalent to [Pu( p1, p0, q1, q0)]1 Pu( p0, p1, q0, q1) and this inequality follows from the fact that a weighted harmonic mean of n positive numbers is equal or less than the corresponding weighted arithmetic mean; see Hardy, Littlewood and Po´ lya (1934, p. 26).

reciprocal stochastic price index deﬁned by the formula (16.59) has no author’s name associated with it but it was noted by Fisher (1922, p. 467) as his index number formula 13.
Vartia (1978, p. 272) called this index the harmonic Laspeyres index and his terminology will be used.

16.89 Now consider the class of symmetrically weighted reciprocal price indices deﬁned as: Pmr( p0, p1, q0, q1) 1 n P i=1 m(s0 i , s1 i ) 1 p1 i p0 i (16:61) i , s1 where, as usual, m(s0 i ) is a homogeneous symmetric mean of the period 0 and 1 expenditure shares on commodity i.
However, none of the indices deﬁned by equations (16.59) to (16.61) satisﬁes the time reversal test.

16.90 The fact that Theil’s index number formula PT satisﬁes the time reversal test leads to a preference for Theil’s ‘‘best’’ weighted stochastic approach.

index as the i =p0 16.91 The main features of the weighted stochastic approach to index number theory can be summarized as follows.

It is ﬁrst necessary to pick two periods and a transactions domain of deﬁnition.
As usual, each value transaction for each of the n commodities in the domain of deﬁnition is split up into price and quantity components.

Then, assuming there are no new commodities or no disappearing commodities, there are n price relatives p1 i pertaining to the two situations under consideration along with the corre- sponding 2n expenditure shares.
The weighted sto- chastic approach just assumes that these n relative prices, or some transformation of these price relatives, f ( p1 i ), have a discrete statistical distribution, where i , s1 the ith probability, ri=m(s0 i ), is a function of the expenditure shares pertaining to commodity i in the two situations under consideration, s0 i and s1 i .

Diﬀer- ent price indices result, depending on how the func- tions f and m are chosen.
In Theil’s approach, the transformation function f is the natural logarithm and the mean function m is the simple unweighted arith- metic mean.

i =p0 16.92 There is a third aspect to the weighted sto- chastic approach to index number theory: it has to be decided what single number best summarizes the dis- tribution of the n (possibly transformed) price rela- tives.
In the above analysis, the mean of the discrete distribution was chosen as the ‘‘best’’ summary mea- sure for the distribution of the (possibly transformed) price relatives; but other measures are possible.

In the weighted median or various trimmed particular, means are often suggested as the ‘‘best’’ measure of central tendency because these measures minimize the inﬂuence of outliers.

Detailed discussion of these alternative measures of central tendency is, however, beyond the scope of this chapter.
Additional material on stochastic approaches to index number theory and references to the literature can be found in Clements and Izan (1981; 1987), Selvanathan and Rao (1994), 303 CONSUMER PRICE INDEX: THEORY AND PRACTICE Diewert (1995b), Cecchetti (1997) and Wynne (1997; 1999).

16.93 Instead of taking the above stochastic approach to index number theory, it is possible to take the same raw data that are used in this approach but use an axiomatic approach.
Thus, in the follow- ing section, the price index is regarded as a value- weighted function of the n price relatives and the test approach to index number theory is used in order to determine the functional form for the price index.
Put the axiomatic approach in the next another way, section looks at the properties of alternative descrip- tive statistics that aggregate the individual price rela- tives (weighted by their economic importance) into summary measures of price change in an attempt to ﬁnd the ‘‘best’’ summary measure of price change.

Thus the axiomatic approach pursued below can be viewed as a branch of the theory of descriptive statistics.
The second axiomatic approach to bilateral price indices The basic framework and some preliminary tests 16.94 As mentioned in paragraphs 16.1 to 16.10, one of Walsh’s approaches to index number theory was an attempt to determine the ‘‘best’’ weighted average of the price relatives, ri.61 This is equivalent to using an axiomatic approach to try to determine the ‘‘best’’ index of the form P(r, v0, v1), where v0 and v1 are the vectors of expenditures on the n commodities during periods 0 and 1.62 Initially, rather than starting with indices of the form P(r, v0, v1), indices of the form P( p0, p1, v0, v1) will be considered, since this framework will be more comparable to the ﬁrst bilateral axiomatic framework taken in paragraphs 16.30 to 16.73.
As will be seen below, if the invariance to changes in the units 61 Fisher also took this point of view when describing his approach to index number theory: their price relatives.

This deﬁnition has, An index number of the prices of a number of commodities is an average for concreteness, been of expressed in terms of prices.
But in like manner, an index number can be calculated for wages, for quantities of goods imported or exported, and, in fact, for any subject matter involving divergent changes of a group of magnitudes.

Again, this deﬁnition has been expressed in terms of time.
But an index number can be applied with equal propriety to comparisons between two places or, in fact, to comparisons between the magnitudes of a group of elements under any one set of circumstances and their mag- nitudes under another set of circumstances (Fisher (1922, p. 3)).
In setting up his axiomatic approach, Fisher imposed axioms on the price and quantity indices written as functions of the two price vectors, p0 and p1, and the two quantity vectors, q0 and q1; i.e., he did not write his price index in the form P(r, v0, v1) and impose axioms on indices of this type.
Of course, in the end, his ideal price index turned out to be the geometric mean of the Laspeyres and Paasche price indices and, as was seen in Chapter 15, each of these indices can be written as expenditure share-weighted avera- ges of the n price relatives, ri p1 62 Chapter 3 in Vartia (1976) considered a variant of this axiomatic approach.

i =ip0 i : 304 of measurement test is imposed on an index of the form P( p0, p1, v0, v1), then P( p0, p1, v0, v1) can be writ- ten in the form P(r, v0, v1).
16.95 Recall that the product test (16.17) was used to deﬁne the quantity index Q( p0, p1, q0, q1):V1/V 0P( p0, p1, q0, q1) that corresponded to the bilateral price index P( p0, p1, q0, q1).
A similar product test holds in the present framework; i.e., given that the functional form for the price index P( p0, p1, v0, v1) has been determined, then the corresponding implicit quantity index can be deﬁned in terms of P as follows: Q( p0, p1, v0, v1) n P i=1 v0 i n P i=1 v1 i (16:62) P( p0, p1, v0, v1) 16.96 In paragraphs 16.30 to 16.73, the price and quantity indices P( p0, p1, q0, q1) and Q( p0, p1, q0, q1) were determined jointly; i.e., not only were axioms imposed on P( p0, p1, q0, q1) but they were also imposed on Q( p0, p1, q0, q1) and the product test (16.17) was used to translate these tests on Q into tests on P. In this section, this approach will not be followed: only tests on P( p0, p1, v0, v1) will be used in order to determine the ‘‘best’’ price index of this form.
Thus there is a parallel theory for quantity indices of the form Q(q0, q1, v0, v1), where it is attempted to ﬁnd the ‘‘best’’ value-weighted average of the quantity relatives, q1 i =q0 i .63 16.97 For the most part, the tests which will be imposed on the price index P( p0, p1, v0, v1) in this section are counterparts to the tests that were imposed on the price index P( p0, p1, q0, q1) in paragraphs 16.30 to 16.73.
It will be assumed that every component of each price and value vector is positive; i.e., pt 0n and vt 0n for is desired to set v0=v1, t=0,1.

If the common expenditure vector is denoted by v; if it is desired to set p0=p1, the common price vector is denoted by p. it 16.98 The ﬁrst two tests are straightforward coun- terparts to the corresponding tests in paragraph 16.34.
T1: Positivity: P( p0, p1, v0, v1) > 0 T2: Continuity: P( p0, p1, v0, v1) is a continuous function of its arguments T3: Identity or constant prices test: P( p, p, v0, v1)=1 That is, if the price of every good is identical during the two periods, then the price index should equal unity, no matter what the value vectors are.

Note that the two value vectors are allowed to be diﬀerent in the above test.
63 It turns out that the price index that corresponds to this ‘‘best’’ quantity index, deﬁned as P(q0, q1, v0, v1) Pn i Q (q0, q1, v0, v1), will not equal the ‘‘best’’ price index, P( p0, p1, v0, v1).

Thus the axiomatic approach used here generates separate ‘‘best’’ price and quantity indices whose product does not equal the value ratio in general.

This is a disadvantage of the second axiomatic approach to bilateral indices compared to the ﬁrst approach studied above.
i = Pn i=1 v0 i=1 v1 THE AXIOMATIC AND STOCHASTIC APPROACHES TO INDEX NUMBER THEORY Homogeneity tests 16.99 The following four tests restrict the behaviour of the price index P as the scale of any one of the four vectors p0, p1, v0, v1 changes.

T4: Proportionality in current prices: P( p0, lp1, v0, v1)=lP( p0, p1, v0, v1) for l > 0 That is, if all period 1 prices are multiplied by the posi- tive number l, then the new price index is l times the old price index.
Put another way, the price index function P( p0, p1, v0, v1) is (positively) homogeneous of degree one in the components of the period 1 price vector p1.

This test is the counterpart to test T5 in paragraph 16.37.
16.100 In the next test, instead of multiplying all period 1 prices by the same number, all period 0 prices are multiplied by the number l. T5: Inverse proportionality in base period prices: P(lp0, p1, v0, v1)=l1P( p0, p1, v0, v1) for l > 0 That is, if all period 0 prices are multiplied by the positive number l, then the new price index is 1/l times the old price index.
Put another way, the price index function P( p0, p1, v0, v1) is (positively) homogeneous of degree minus one in the components of the period 0 price vector p0.

This test is the counterpart to test T6 in paragraph 16.39.

16.101 The following two homogeneity tests can also be regarded as invariance tests.
T6: Invariance to proportional changes in current period values : P( p0, p1, v0, lv1)=P( p0, p1, v0, v1) for all l > 0 That is, if current period values are all multiplied by the number l, then the price index remains unchanged.

Put another way, the price index function P( p0,p1,v0,v1) is (positively) homogeneous of degree zero in the compo- nents of the period 1 value vector v1.

two price vectors p0 and p1 and the two vectors of expenditure shares, s0 and s1.
16.103 Walsh (1901, p. 104) suggested the spirit of tests T6 and T7 as the following quotation indicates: ‘‘What we are seeking is to average the variations in the exchange value of one given total sum of money in relation to the several classes of goods, to which several variations [i.e., the price relatives] must be assigned weights propor- tional to the relative sizes of the classes.
Hence the relative sizes of the classes at both the periods must be considered.’’ 16.104 Walsh also realized that weighting the ith price relative ri by the arithmetic mean of the value weights in the two periods under consideration, (1/2) [v0 i ] would give too much weight to the expenditures of the period that had the highest level of prices: i +v1 At ﬁrst sight it might be thought suﬃcient to add up the weights of every class at the two periods and to divide by two.
This would give the (arithmetic) mean size of every class over the two periods together.

But such an operation is manifestly wrong.


In the ﬁrst place, the sizes of the classes at each period are reckoned in the money of the period, and if it happens that the exchange value of money has fallen, or prices in general have risen, greater inﬂuence upon the result would be given to the weighting of the second period; or if prices in general have fallen, greater inﬂuence would be given to the weighting of the ﬁrst period.
Or in a comparison between two countries, greater inﬂuence would be given to the weighting of the country with the higher level of prices.

But it is plain that the one period, or the one country, is as important, in our comparison between them, as the other, and the weighting in the averaging of their weights should really be even (Walsh (1901, pp.

104–105)).
104–105)).

16.105 As a solution to the above weighting prob- lem, Walsh (1901, p. 202; 1921a, p. 97) proposed the following geometric price index: PGW ( p0, p1, v0, v1) w(i) p1 i p0 i n Y i=1 (16:64) where the ith weight in the above formula was deﬁned as i v1 (v0 i )1=2 kv1 k)1=2 (v0 n P k=1 = i s1 (s0 i )1=2 ks1 k)1=2 (s0 n P k=1 i=1, .

.

.
.

.

, n (16:65) T7: Invariance to proportional changes in base period values: w(i) P( p0, p1, lv0, v1)=P( p0, p1, v0, v1) for all l > 0 That is, if base period values are all multiplied by the number l, then the price index remains unchanged.

Put another way, the price index function P( p0, p1, v0, v1) is (positively) homogeneous of degree zero in the compo- nents of the period 0 value vector v0.
16.102 T6 and T7 together impose the property that the price index P does not depend on the absolute mag- nitudes of the value vectors v0 and v1.

Using test T6 with l=1=Pn i and using test T7 with l=1=Pn i , it can be seen that P has the following property: i=1v1 i=1v0 P( p0, p1, v0, v1)=P( p0, p1, s0, s1) (16:63) where s0 and s1 are the vectors of expenditure shares for is periods 0 and 1; i=Pn i vt st k for t=0,1.
Thus the tests T6 and T7 imply that the price index function P is a function of the the ith component of st k=1vt i.e., The second equation in (16.65) shows that Walsh’s geometric price index PGW ( p0, p1, v0, v1) can also be written as a function of the expenditure share vectors, s0 and s1; i.e., PGW ( p0, p1, v0, v1) is homogeneous of degree zero in the components of the value vectors v0 and v1 and so PGW ( p0, p1, v0, v1)=PGW ( p0, p1, s0, s1).
Thus Walsh came very close to deriving the To¨ rnqvist–Theil index deﬁned earlier by equation (16.48).64 64 Walsh’s index could be derived using the same arguments as Theil, i )1=2 except that the geometric average of the expenditure shares (s0 could be taken as a preliminary probability weight for the ith loga- rithmic price relative, ln ri.

These preliminary weights are then nor- malized to add up to unity by dividing by their sum.
It is evident that Walsh’s geometric price index will closely approximate Theil’s index using normal time series data.

More formally, regarding both p0, p1, v0, v1, shown that indices PW( p0, p1, v0, v1) approximates PT( p0, p1, v0, v1) to the second order around an equal price (i.e., p0=p1) and quantity (i.e., q0=q1) point.
functions of can be i s1 as it 305 CONSUMER PRICE INDEX: THEORY AND PRACTICE Invariance and symmetry tests 16.106 The next ﬁve tests are invariance or symmetry tests and four of them are direct counterparts to similar tests in paragraphs 16.42 to 16.46 above.

The ﬁrst invariance test is that the price index should remain unchanged if the ordering of the commodities is changed.
T8: Commodity reversal test (or invariance to changes in the ordering of commodities): P ( p0*, p1*, v0*, v1*)=P( p0, p1, v0, v1) where pt* denotes a permutation of the components of the vector pt and vt* denotes the same permutation of the components of vt for t=0,1.

16.107 The next test asks that the index be invariant to changes in the units of measurement.

T9: Invariance to changes in the units of measurement (commensurability test): P(a1p0 1, .

.

.

, anp0 1, .

.

.

, p0 n; a1p1 1, .

.

.


, anp1 n; v0 n; p1 P( p0 for all a1 > 0, .

.

.

, an > 0 1, .

.

.

, p1 n; v0 1, .

.

.

, v0 1, .

.

.

, v0 n; v1 1, .

.

.

, v1 n; v1 1, .

.

.

, v1 n) n)= That is, the price index does not change if the units of measurement for each commodity are changed.

Note t, that the expenditure on commodity i during period t, vi does not change if the unit by which commodity i is measured changes.

16.108 The last test has a very important implica- tion.
Let a1=1=p0 n and substitute these values for the ai into the deﬁnition of the test.

The fol- lowing equation is obtained: 1, .

.

.

, an=1=p0 P( p0, p1, v0, v1)=P(1n, r, v0, v1) P*(r, v0, v1) (16:66) where 1n is a vector of ones of dimension n and r is a i.e., the ith component vector of the price relatives; of r is ri p1 i =p0 i .
Thus, if the commensurability test T9 is satisﬁed, then the price index P( p0, p1, v0, v1), which is a function of 4n variables, can be written as a function of 3n variables, P*(r, v0, v1), where r is the vector of price relatives and P*(r, v0, v1) is deﬁned as P(1n, r, v0, v1).

16.109 The next test asks that the formula be invariant to the period chosen as the base period.
T10: Time reversal test: P( p0, p1, v0, v1)=1=P( p1, p0, v1, v0) That is, if the data for periods 0 and 1 are interchanged, then the resulting price index should equal the reciprocal of the original price index.

Obviously, in the one good case when the price index is simply the single price ratio, this test will be satisﬁed (as are all the other tests listed in this section).
16.110 The next test is a variant of the circularity test, introduced in paragraphs 15.76 to 15.97 of Chapter 15.65 65 See equation (15.77) in Chapter 15.

306 T11: Transitivity in prices for fixed value weights: P( p0, p1, vr, vs)P( p1, p2, vr, vs)=P( p0, p2, vr, vs) In this test, the expenditure weighting vectors, vr and vs, are held constant while making all price comparisons.
Given that these weights are held constant, however, the test asks that the product of the index going from period 0 to 1, P( p0, p1, vr, vs), times the index going from period 1 to 2, P( p1, p2, vr, vs), should equal the direct index that compares the prices of period 2 with those of period 0, P( p0, p2, vr, vs).

Obviously, this test is a many- commodity counterpart to a property that holds for a single price relative.
16.111 The ﬁnal test in this section captures the idea that the value weights should enter the index number formula in a symmetric manner.

T12: Quantity weights symmetry test: P( p0, p1, v0, v1)=P( p0, p1, v1, v0) That is, if the expenditure vectors for the two periods are interchanged, then the price index remains invariant.
This property means that, if values are used to weight the prices in the index number formula, then the period 0 values v0 and the period 1 values v1 must enter the formula in a symmetric or even-handed manner.

A mean value test 16.112 The next test is a mean value test.

T13: Mean value test for prices: mini ( p1 i =p0 maxi ( p1 i : i=1, .

.

.

, n) P( p0, p1, v0, v1) i =p0 i : i=1, .

.

.
.

.

, n) (16:67) That is, the price index lies between the minimum price ratio and the maximum price ratio.

Since the price index is to be interpreted as an average of the n price ratios, p1 i =p0 i , it seems essential that the price index P satisfy this test.

Monotonicity tests 16.113 The next two tests in this section are mono- tonicity tests; index P( p0, p1, v0, v1) change as any component of the two price vectors p0 and p1 increases?
i.e., how should the price T14: Monotonicity in current prices: P( p0, p1, v0, v1) < P( p0, p2, v0, v1) if p1 < p2 That is, if some period 1 price increases, then the price index must increase (holding the value vectors ﬁxed), so that P( p0, p1, q0, q1) is increasing in the components of p1 for ﬁxed p0, v0 and v1.
T15: Monotonicity in base prices: P( p0, p1, v0, v1) > P( p2, p1, v0, v1) if p0 < p2 That is, if any period 0 price increases, then the price index must decrease, so that P( p0, p1, q0, q1) is decreas- ing in the components of p0 for ﬁxed p1, v0 and v1.
THE AXIOMATIC AND STOCHASTIC APPROACHES TO INDEX NUMBER THEORY Weighting tests 16.114 The above tests are not suﬃcient to deter- mine the functional form of the price index; for example, it can be shown that both Walsh’s geometric price index PGW( p0, p1, v0, v1) deﬁned by equation (16.65) and the index PT ( p0, p1, v0, v1) deﬁned by To¨ rnqvist–Theil equation (16.48) satisfy all of the above axioms.
Thus, at least one more test will be required in order to determine the functional form for the price index P( p0, p1, v0, v1).

16.115 The tests proposed thus far do not specify exactly how the expenditure share vectors s0 and s1 are to be used in order to weight, say, the ﬁrst price relative, p1 1=p0 1.

The next test says that only the expenditure shares s0 1 pertaining to the ﬁrst commodity are to be used in order to weight the prices that correspond to commodity 1, p1 1 and s1 1 and p0 1.
T16: Own share price weighting: 1, 1, .

.

.

, 1; v0, v1) P( p0 1, 1, .

.

.
.

.

, 1; p1 =f 1, v0 1 1, p1 p0 , v1 1 v0 k n P k=1 k equals st (16:68) n P k=1 v1 k 1=Pn k=1vt Note that vt 1, the expenditure share for commodity 1 in period t. The above test says that if all the prices are set equal to 1 except the prices for commodity 1 in the two periods, but the expenditures in the two periods are arbitrarily given, then the index depends only on the two prices for commodity 1 and the two expenditure shares for commodity 1.
The axiom says that a function of 2+2n variables is actually only a function of four variables.66 16.116 Of course, if test T16 is combined with test T8, the commodity reversal test, then it can be seen that P has the following property: P(1, .

.

.

, 1, p0 =f i , p1 p0 i , 1, .

.

.

, 1;1, .

.

.

, 1, p1 n P k=1 i , v0 i , v1 i i , 1, .

.

.

, 1; v0; v1) n P k=1 v1 k v0 k i=1, .

.

.
.

.

, n (16:69) Equation (16.69) says that, if all the prices are set equal to 1 except the prices for commodity i in the two peri- ods, but the expenditures in the two periods are arbi- trarily given, then the index depends only on the two prices for commodity i and the two expenditure shares for commodity i.

16.117 The ﬁnal test that also involves the weighting of prices is the following one: T17: Irrelevance of price change with tiny value weights: P( p0 1, 1, .

.

.

, 1; p1 1, 1, .

.

.


, 1; 0; v0 2, .

.

.

, v0 n; 0; v1 2, .

.

.

, v1 n)=1 (16:70) 66 In the economics literature, axioms of this type are known as separability axioms.
The test T17 says that, if all the prices are set equal to 1 except the prices for commodity 1 in the two periods, and the expenditures on commodity 1 are 0 in the two periods but the expenditures on the other commodities are arbitrarily given, then the index is equal to 1.67 Thus, roughly speaking, if the value weights for commodity 1 are tiny, then it does not matter what the price of commodity 1 is during the two periods.
16.118 Of course, if test T17 is combined with test T8, the commodity reversal test, then it can be seen that P has the following property: for i=1, .

.

.

, n: P(1, .

.

.

, 1, p0 n; v1 v0 i , 1, .

.

.

, 1; 1, .

.

.

, 1, p1 n)=1 1, .

.

.

, 0, .

.

.

, v1 i , 1, .

.

.

, 1; v0 1, .

.

.

, 0, .

.

.


, (16:71) Equation (16.71) says that, if all the prices are set equal to 1 except the prices for commodity i in the two periods, and the expenditures on commodity i are 0 during the two periods but the other expenditures in the two periods are arbitrarily given, then the index is equal to 1.

16.119 This completes the listing of tests for the approach to bilateral index number theory based on the weighted average of price relatives.
It turns out that the above tests are suﬃcient to imply a speciﬁc functional form for the price index, as seen in the next section.
16.120 The To¨ rnqvist–Theil price index and the second test approach to bilateral indices In Appendix 16.1 to this chapter, it is shown that, if the number of commodities n exceeds two and the bilateral price index function P( p0, p1, v0, v1) satisﬁes the 17 axioms listed above, then P must be the To¨ rnqvist– Theil price index PT( p0, p1, v0, v1) deﬁned by equation (16.48).68 Thus the 17 properties or tests listed in para- graphs 16.94 to 16.129 provide an axiomatic character- ization of the To¨ rnqvist–Theil price index, just as the 20 tests listed in paragraphs 16.30 to 16.73 provided an axiomatic characterization of the Fisher ideal price index.
16.121 Obviously, there is a parallel axiomatic the- ory for quantity indices of the form Q(q0, q1, v0, v1) that depend on the two quantity vectors for periods 0 and 1, q0 and q1, as well as on the corresponding two expen- diture vectors, v0 and v1.
Thus, if Q(q0, q1, v0, v1) satisﬁes the quantity counterparts to tests T1 to T17, then Q must be equal to the To¨ rnqvist–Theil quantity index 1 and v1 67 Strictly speaking, since all prices and values are required to be positive, the left-hand side of equation (16.70) should be replaced by the limit as the commodity 1 values, v0 68 The To¨ rnqvist–Theil price index satisﬁes all 17 tests, but the proof in Appendix 16.1 does not use all these tests to establish the result in the opposite direction: tests 5, 13, 15 and one of 10 or 12 were not required in order to show that an index satisfying the remaining tests must be the To¨ rnqvist–Theil price index.
For alternative characterizations of the To¨ rnqvist–Theil price index, see Balk and Diewert (2001) and Hillinger (2002).

1, approach 0.

307 CONSUMER PRICE INDEX: THEORY AND PRACTICE QT (q0, q1, v0, v1) deﬁned, as follows: ln QT (q0, q1, v0, v1) P n i=1 1 2 (s0 i +s1 i ) ln q1 i q0 i (16:72) where as usual, the period t expenditure share on com- modity i, st k for i=1, .

.

.
.

.

, n i, and t=0, 1: is deﬁned as vt i=Pn k=1vt 16.122 Unfortunately, the implicit To¨ rnqvist–Theil price index, PIT(q0, q1, v0, v1) that corresponds to the To¨ rnqvist–Theil quantity index QT, deﬁned by equation (16.72) using the product test, is not equal to the direct To¨ rnqvist–Theil price index PT( p0, p1, v0, v1), deﬁned by equation (16.48).
The product test equation that deﬁnes PIT in the present context is given by the following equation: PIT (q0, q1, v0, v1) n P i=1 v0 i n P i=1 v1 i (16:73) QT (q0, q1, v0, v1) The fact that the direct To¨ rnqvist–Theil price index PT is not in general equal to the implicit To¨ rnqvist–Theil price index PIT, deﬁned by equation (16.73), is something of a disadvantage compared to the axiomatic approach outlined in paragraphs 16.30 to 16.73, which led to the Fisher ideal price and quantity indices being considered ‘‘best’’.
Using the Fisher approach meant that it was not necessary to decide whether the aim was to ﬁnd a ‘‘best’’ price index or a ‘‘best’’ quantity index: the theory out- lined in paragraphs 16.30 to 16.73 determined both indices simultaneously.

In the To¨ rnqvist–Theil approach outlined in this section, however, it is necessary to choose between a ‘‘best’’ price index or a ‘‘best’’ quantity index.69 16.123 Other tests are of course possible.
A coun- terpart to Test T16 in paragraph 16.49, the Paasche and is the following geometric Laspeyres bounding test, Paasche and Laspeyres bounding test: PGL( p0, p1, v0, v1) P( p0, p1, v0, v1) PGP( p0, p1, v0, v1) or PGP( p0, p1, v0, v1) P( p0, p1, v0, v1) PGL( p0, p1, v0, v1) (16:74) where the logarithms of the geometric Laspeyres and geometric Paasche price indices, PGL and PGP, are deﬁned as follows: ln PGL( p0, p1, v0, v1) P n i=1 ln PGP( p0, p1, v0, v1) P n i=1 s0 i ln p1 i p0 i s1 i ln p1 i p0 i (16:75) (16:76) As usual, the period t expenditure share on commodity i, i, is deﬁned as vt st k for i=1, .
.

.

, n and t=0, 1.

It can be shown that the To¨ rnqvist–Theil price index 1= Pn k=1 vt test, but the geometric Walsh price PT ( p0, p1, v0, v1) deﬁned by equation (16.48) satisﬁes index this PGW ( p0, p1, v0, v1) deﬁned by equation (16.65) does not.
The geometric Paasche and Laspeyres bounding test was not included as a primary test in this section because it was not known a priori what form of averaging of the price relatives (e.g., geometric or arithmetic or har- monic) would turn out to be appropriate in this test framework.
The test (16.74) is an appropriate one if it has been decided that geometric averaging of the price relatives is the appropriate framework, since the geo- metric Paasche and Laspeyres indices correspond to ‘‘extreme’’ forms of value weighting in the context of geometric averaging and it is natural to require that the ‘‘best’’ price index lies between these extreme indices.
16.124 Walsh (1901, p. 408) pointed out a problem with his geometric price index deﬁned by equation (16.65), which also applies to the To¨ rnqvist–Theil price index, PT ( p0, p1, v0, v1), deﬁned by equation (16.48): these geometric type indices do not give the ‘‘right’’ answer when the quantity vectors are constant (or proportional) over the two periods.
In this case, Walsh thought that the ‘‘right’’ answer must be the Lowe index, which is the ratio of the costs of purchasing the constant basket during the two periods.

Put another way, the geometric indices PGW and PT do not satisfy the ﬁxed basket test T4 in para- graph 16.35.

What then was the argument that led Walsh to deﬁne his geometric average type index PGW?

It turns out that he was led to this type of index by considering another test, which will now be explained.

16.125 Walsh (1901, pp.


228–231) derived his test by considering the following very simple framework.

Let there be only two commodities in the index and suppose that the expenditure share on each commodity is equal in each of the two periods under consideration.
The price index under these conditions is equal to 1)=P*(r1, r2;1/2, 1/2;1/2, 1/2): P ( p1 m(r1, r2), where m(r1, r2) is a symmetric mean of the 2.70 In this two price relatives, r1 p1 1=p0 framework, Walsh then proposed the following price relative reciprocal test: 1 and r2 p1 2=p0 0, p2 1, p2 0, v2 1, v2 0;p1 0;v1 1;v1 m(r1, r1 1 )=1 (16:77) Thus, if the value weighting for the two commodities is equal over the two periods and the second price relative is the reciprocal of the ﬁrst price relative r1, then Walsh (1901, p. 230) argued that the overall price index under these circumstances ought to equal 1, since the relative fall in one price is exactly counterbalanced by a rise in the other and both commodities have the same expenditures in each period.
He found that the geometric mean satis- ﬁed this test perfectly but the arithmetic mean led to index values greater than 1 (provided that r1 was not equal to 1) and the harmonic mean led to index values that were less than 1, a situation which was not at all satisfactory.71 Thus he was led to some form of geometric 69 Hillinger (2002) suggested taking the geometric mean of the direct and implicit To¨ rnqvist–Theil price indices in order to resolve this conﬂict.
Unfortunately, the resulting index is not ‘‘best’’ for either set of axioms that were suggested in this section.

70 Walsh considered only the cases where m was the arithmetic, geo- metric and harmonic means of r1 and r2.

71‘‘This tendency of the arithmetic and harmonic solutions to run into the ground or to ﬂy into the air by their excessive demands is clear indication of their falsity’’ (Walsh (1901, p. 231)).
308 THE AXIOMATIC AND STOCHASTIC APPROACHES TO INDEX NUMBER THEORY averaging of the price relatives in one of his approaches to index number theory.

16.126 A generalization of Walsh’s result is easy to the mean function, m(r1, r2), obtain.

Suppose that satisﬁes Walsh’s reciprocal test (16.77) and, in addition, m is a homogeneous mean, so that it satisﬁes the fol- lowing property for all r1>0, r2>0 and l>0: m(lr1, lr2)=lm(r1, r2): (16:78) Let r1>0, r2>0.
Then m(r1, r2)= r1 r1 m(r1, r2) using (16:78) with l= =r1m r1 r1 =r1m 1, , r2 r1 r2 r1 =r1 f r2 r1 1 r1 (16:79) where the function of one (positive) variable f(z) is deﬁned as f (z) m(1, z) (16:80) Using equation (16.77): 1=m(r1, r1 1 ) r1 r1 = m(r1, r1 1 ) =r1m(1, r2 1 ) using (16:78) with l= 1 r1 (16:81) Using equation (16.80), equation (16.81) can be re- arranged in the following form: f (r2 1 )=r1 1 (16:82) Letting z r2 1 becomes: so that z1=2=r1 1 , equation (16.82) f (z)=z1=2 (16:83) Now substitute equation (16.83) into equation (16.79) and the functional form for the mean function m(r1, r2) is determined: i or p1 i i or p1 i the market, becoming a ‘free good’, that fact ought not to render the index number for 1910 zero.’’ In the present context, this test could be interpreted as the if any single price p0 tends to following one: zero, then the price index P( p0, p, v0, v1) should not tend to zero or plus inﬁnity.
However, with this interpretation of the test, which regards the values vt i as remaining constant as the p0 tends to zero, none of the commonly used index number formulae would satisfy this test.

Hence this test should be that applies to price indices interpreted as a test P( p0, p1, q0, q1) of the type studied in paragraphs 16.30 to 16.73, which is how Fisher intended the test to apply.
Thus, Fisher’s price determinateness test should i or p1 be interpreted as follows: if any single price p0 i then the price index P( p0, p, q0, q1) tends to zero, should not tend to zero or plus inﬁnity.

With this interpretation of the test, it can be veriﬁed that Las- peyres, Paasche and Fisher indices satisfy this test but the To¨ rnqvist–Theil price index does not.
Thus, when using the To¨ rnqvist–Theil price index, care must be taken to bound the prices away from zero in order to avoid a meaningless index number value.

16.128 Walsh was aware that geometric average type indices such as the To¨ rnqvist–Theil price index PT or Walsh’s geometric price index PGW deﬁned by equation (16.64) become somewhat unstable72 as indi- vidual price relatives become very large or small: Hence in practice the geometric average is not likely to depart much from the truth.
Still, we have seen that when the classes [i.e., expenditures] are very unequal and the price variations are very great, this average may deﬂect considerably (Walsh (1901, p. 373)).

In the cases of moderate inequality in the sizes of the classes and of excessive variation in one of the prices, there seems to be a tendency on the part of the geometric method to deviate by itself, becoming untrustworthy, while the other two methods keep fairly close together (Walsh (1901, p. 404)).
16.129 Weighing all the arguments and tests pre- sented above, it seems that there may be a slight pre- ference for the use of the Fisher ideal price index as a suitable target index for a statistical agency, but, of course, opinions may diﬀer on which set of axioms is the most appropriate to use in practice.
m(r1, r2)=r1 f r2 r1 =r1 1=2 r2 r1 = r1=2 1 r1=2 2 (16:84) The test properties of the Lowe and Young indices Thus, the geometric mean of the two price relatives is the only homogeneous mean that will satisfy Walsh’s price relative reciprocal test.

16.127 There is one additional test that should be mentioned.

Fisher (1911, p. 401) introduced this test in his ﬁrst book that dealt with the test approach to index number theory.
He called it the test of determi- nateness as to prices and described it as follows: ‘‘A price index should not be rendered zero, inﬁnity, or indeterminate by an individual price becoming zero.

Thus, if any commodity should in 1910 be a glut on 16.130 The Young and Lowe indices were deﬁned in Chapter 15.

In the present section, the axiomatic prop- erties of these indices with respect to their price argu- ments are developed.73 16.131 Let qb [qb 1, .

.

.

, pb 1, .

.

.


, qb n] denote the quantity and price vectors pertaining to some base year.

The corresponding base year expenditure n] and pb [pb 72 That is, the index may approach zero or plus inﬁnity.

73 Baldwin (1990, p. 255) worked out a few of the axiomatic properties of the Lowe index.

309 CONSUMER PRICE INDEX: THEORY AND PRACTICE shares can be deﬁned in the usual way as sb i i qb pb i n P kqb pb k k=1 i=1, .

.

.

, n (16:85) 1, .

.

.


, sb Let sb [sb n] denote the vector of base year expenditure shares.
The Young (1812) price index between periods 0 and t is deﬁned as follows: T5: Homogeneity test for period 0 prices: P(lp0, pt)=l1P( p0, pt) for all l > 0 T6: Commodity reversal test: P( pt, p0)=P( p0*, pt*) where p0* and pt*denote the same permutation of the components of the price vectors p0 and pt75 T7: Invariance to changes in the units of measurement (commensurability test) PY ( p0, pt, sb) P n i=1 sb i pt i p0 i (16:86) T8: Time reversal test: P( pt, p0)=1=P( p0, pt) The Lowe (1823, p. 316) price index74 between periods 0 and t is deﬁned as follows: T9: Circularity or transitivity test: P( p0, p2)=P( p0, p1)P( p1, p2) PLo( p0, pt, qb) n P i=1 n P k=1 iqb pt i kqb p0 k = n P i=1 n P k=1 sb i sb k pt i pb i !
p0 k pb k (16:87) 16.132 Drawing on the axioms listed above in this chapter, 12 desirable axioms for price indices of the form P( p0, p1) are listed below.

The period 0 and t price vectors, p0 and pt, are presumed to have strictly positive components.
T1: Positivity: P( p0, pt) > 0 if all prices are positive T2: Continuity: P( p0, pt) is a continuous function of prices T3: Identity test: P( p0, p0)=1 T4: Homogeneity test for period t prices: P( p0, lpt)=lP( p0, pt) for all l > 0 T10: Mean value test: min fpt P( pt, p0) max fpt i=p0 i : i=1, .

.

.

, ng i=p0 i : i=1, .

.

.


, ng T11: Monotonicity test with respect to period t prices: P( p0, pt) < P( p0, pt*) if pt < pt* T12: Monotonicity test with respect to period 0 prices: P( p0, pt) > P( p0*, pt) if p0 < p0* 16.133 It is straightforward to show that the Lowe index deﬁned by equation (16.87) satisﬁes all 12 of the axioms or tests listed above.
Hence the Lowe index has very good axiomatic properties with respect to its price variables.76 16.134 It is straightforward to show that the Young index deﬁned by equation (16.86) satisﬁes 10 of the 12 axioms, failing the time reversal test T8 and the circu- larity test T9.

Thus the axiomatic properties of the Young index are deﬁnitely inferior to those of the Lowe index.

74 This index number formula is also precisely Bean and Stine’s (1924, p. 31) Type A index number formula.
Walsh (1901, p. 539) initially mistakenly attributed Lowe’s formula to G. Poulett Scrope (1833), who wrote Principles of Political Economy in 1833 and suggested Lowe’s formula without acknowledging Lowe’s priority.

But in his discussion of Fisher’s (1921) paper, Walsh (1921b, p. 543–544) corrects his mistake on assigning Lowe’s formula: What index number should you then use?

It should be this: Pqp1=Pqp0.

This is the method used by Lowe within a year or two of one hundred years ago.
In my [1901] book, I called it Scrope’s index number; but it should be called Lowe’s.

Note that in it are used quantities neither of a base year nor of a subsequent year.

The quantities used should be rough estimates of what the quantities were throughout the period or epoch.

310 75 In applying this test to the Lowe and Young indices, it is assumed that the base year quantity vector qb and the base year share vector sb are subject to the same permutation.
76 From the discussion in Chapter 15, it will be recalled that the main problem with the Lowe index occurs if the quantity weight vector qb is not representative of the quantities that were purchased during the time interval between periods 0 and 1.

THE AXIOMATIC AND STOCHASTIC APPROACHES TO INDEX NUMBER THEORY Appendix 16.1 Proof of the optimality of the To¨ rnqvist–Theil price index in the second bilateral test approach The tests (T1, T2, etc.)
mentioned in this appendix are those presented in paragraphs 16.98 to 16.119. ri p1 1.

Deﬁne for i=1, .

.

.

, n. Using T1, T9 and equation (16.66), P( p0, p1, v0, v1)=P*(r, v0, v1).

Using T6, T7 and equation (16.63): i =p0 i P( p0, p1, v0, v1)=P*(r, s0, s1) (A16:1:1) where st is the period t expenditure share vector for t=0, 1.

2.

Let x:(x1, .

.

.

, xn) and y:(y1, .

.

.

, yn) be strictly positive vectors.


The transitivity test T11 and equation the function P* has the following (A16.1.1) property: imply that P*(x; s0, s1)P*(y; s0, s1)=P*(x1y1, .

.

.

, xnyn; s0, s1): (A16:1:2) test T1, P*(r, s0, s1)>0 and using test T14, 3.
Using P*(r, s0, s1) is strictly increasing in the components of r. The identity test T3 implies that P*(1n, s0, s1)=1 where 1n is a vector of ones of dimension n. Using a result attributable to Eichhorn (1978, p. 66), it can be seen that these properties of P* are suﬃcient to imply that there exist positive functions ai(s0, s1) for i=1, .

.

.

, n such that P* has the follow- ing representation: (A16:1:3) n ln P*(r, s0, s1)=P i=1 ai(s0, s1) ln ri (A16:1:4) 4.
The continuity test T2 implies that the positive func- tions ai (s0, s1) are continuous.
For l>0, the linear homo- geneity test T4 implies that ai(s0,s1) ln lri ln P*(lr, s0, s1)=ln l+ ln P*(r,s0, s1) n =P i=1 n =P i=1 n =P i=1 using (A16:1:4) ai(s0,s1) ln l+P n i=1 ai(s0,s1) ln l+ ln P*(r, s0,s1) using (A16:1:4) ai(s0, s1) lnri (A16:1:5) Equating the right-hand sides of the ﬁrst and last lines in equation (A16.1.5) shows that the functions ai(s0, s1) must satisfy the following restriction: n P i=1 ai(s0, s1)=1 (A16:1:6) for all strictly positive vectors s0 and s1.

5.
5.

Using the weighting test T16 and the commodity reversal test T8, equations (16.69) hold.

Equation (16.69) combined with the commensurability test T9 implies that P* satisﬁes the following equation: P*(1, .

.

.

, 1, ri, 1, .

.

.

, 1;s0, s1)=f (1, ri, s0, s1); i=1, .

.

.

, n (A16:1:7) for all ri>0 where f is the function deﬁned in test T16.

6.

Substitute equation (A16.1.7) into equation (A16.1.4) in order to obtain the following system of equations: P*(1, .

.

.

, 1, ri, 1, .

.

.


, 1;s0, s1)=f (1, ri, s0, s1)=ai(s0, s1) ln ri i=1, .

.

.

, n (A16:1:8) But equation (A16.1.8) implies that the positive continuous function of 2n variables ai(s0, s1) is constant with respect to all of its arguments except s0 i and this property holds for each i.

Thus each ai(s0, s1) can be replaced by the positive i ) for i=1, .

.

.

, n.77 continuous function of two variables bi(s0 Now replace the ai (s0, s1) in equation (A16.1.4) by the bi(s0 i , s1 i ) for i=1, .

.

.
.

.

, n and the following representation for P* is obtained: i and s1 i , s1 n ln P*(r, s0, s1)=P i=1 bi(s0 i , s1 i ) ln ri (A16:1:9) 7.

Equation (A16.1.6) implies that the functions bi(s0 i , s1 i ) also satisfy the following restrictions: n P i=1 n i=1 i =1; andP s0 i =1 impliesP s1 bi(s0 i , s1 i )=1 (A16:1:10) n i=1 8.

Assume that the weighting test T17 holds and sub- stitute equation (16.71) into equation (A16.1.9) in order to obtain the following equation: bi(0, 0) ln =0; i=1, .

.

.
.

.

, n (A16:1:11) p1 i p0 i Since the p1 seen that equation (A16.1.11) implies i and p0 i can be arbitrary positive numbers, it can be bi(0, 0)=0; i=1, .

.

.

, n (A16:1:12) that the number of commodities n is equal 9.

Assume (A16.1.10) and than 3.

Using equations to or greater (A16.1.12), Theorem 2 in Acze´ l (1987, p. 8) can be applied and the following functional form for each of the bi(s0 i ) is obtained: i , s1 bi(s0 i , s1 i )=gs0 i +(1 g)s1 i ; i=1, .

.

.
.

.

, n (A16:1:13) where g is a positive number satisfying 0<g<1.

10.

Finally, the time reversal test T10 or the quantity weights symmetry test T12 can be used to show that g must equal ½.
Substituting this value for g back into equation (A16.1.13) and then substituting that equation back into equation (A16.1.9), the functional form for P* and hence P is determined as n 1 ln P( p0, p1, v0, v1)= ln P*(r, s0, s1)=P 2 i=1 (s0 i +s1 i ) ln p1 i : p0 i (A16:1:14) 77 More explicitly, b1(s0 1) a1(s0 1, 1, .

.

.

, 1;s1 1, 1, .

.

.

, 1) and so on.

That is, in deﬁning b1(s0 1, 1, .

.

.

, 1;s1 1), the function a1(s0 1, 1, .

.

.


, 1) is used where all components of the vectors s0 and s1 except the ﬁrst are set equal to an arbitrary positive number such as 1.

1, s1 1, s1 311 THE ECONOMIC APPROACH TO INDEX NUMBER THEORY: THE SINGLE-HOUSEHOLD CASE 17 Introduction 17.1 This chapter and the next cover the econo- mic approach to index number theory.

This chapter considers the case of a single household, while the fol- lowing chapter deals with the case of many households.
A brief outline of the contents of the present chapter follows.

17.2 In paragraphs 17.9 to 17.17, the theory of the cost of living index for a single consumer or household is presented.

This theory was originally developed by the Russian economist, A.A. Konu¨ s (1924).

The relation- ship between the (unobservable) true cost of living index and the observable Laspeyres and Paasche indices will be explained.
It should be noted that, in the economic approach to index number theory, it is assumed that households regard the observed price data as given, while the quantity data are regarded as solutions to various economic optimization problems.

Many price statisticians ﬁnd the assumptions made in the economic approach to be somewhat implausible.
Per- haps the best way to regard the assumptions made in the economic approach is that these assumptions simply formalize the fact that consumers tend to pur- chase more of a commodity if its price falls relative to other prices.

17.3 In paragraphs 17.18 to 17.26, the preferences of the consumer are restricted compared to the completely general case treated in paragraphs 17.9 to 17.17.
In paragraphs 17.18 to 17.26, it is assumed that the func- tion that represents the consumer’s preferences over alternative combinations of commodities is homo- geneous of degree one.

This assumption means that each indiﬀerence surface (the set of commodity bundles that give the consumer the same satisfaction or utility) is a radial blow-up of a single indiﬀerence surface.

With this extra assumption, the theory of the true cost of living simpliﬁes, as will be seen.
17.4 In the sections starting with paragraphs 17.27, 17.33 and 17.44, it is shown that the Fisher, Walsh and To¨ rnqvist price indices (which emerge as being ‘‘best’’ in the various non-economic approaches) are also among the ‘‘best’’ in the economic approach to index number theory.

In these sections, the preference func- tion of the single household will be further restricted compared to the assumptions on preferences made in the previous two sections.
Speciﬁc functional forms for the consumer’s utility function are assumed and it turns out that, with each of these speciﬁc assumptions, the con- sumer’s true cost of living index can be exactly calcu- lated using observable price and quantity data.
Each of the three speciﬁc functional forms for the consumer’s utility function has the property that it can approximate an arbitrary linearly homogeneous function to the sec- ond order; i.e., in economics terminology, each of these three functional forms is ﬂexible.

Hence, using the ter- minology introduced by Diewert (1976), the Fisher, Walsh and To¨ rnqvist price indices are examples of superlative index number formulae.
17.5 In paragraphs 17.50 to 17.54, it is shown that the Fisher, Walsh and To¨ rnqvist price indices approximate each other very closely using ‘‘normal’’ time series data.

This is a very convenient result since these three index number formulae repeatedly show up as being ‘‘best’’ in all the approaches to index number theory.

Hence this approximation result implies that it normally will not matter which of these three indices is chosen as the preferred target index for a consumer price index (CPI).
17.6 The Paasche and Laspeyres price indices have a very convenient mathematical property: they are con- sistent in aggregation.

For example, if the Laspeyres formula is used to construct sub-indices for, say, food or clothing, then these sub-index values can be treated as sub-aggregate price relatives and, using the expenditure shares on these sub-aggregates, the Laspeyres formula can be applied again to form a two-stage Laspeyres price index.
Consistency in aggregation means that this two- stage index is equal to the corresponding single-stage index.

In paragraphs 17.55 to 17.60, it is shown that the superlative indices derived in the earlier sections are not exactly consistent in aggregation but are approximately consistent in aggregation.

17.7 In paragraphs 17.61 to 17.64, a very interesting index number formula is derived: the Lloyd (1975) and Moulton (1996a) price index.
This index number formula makes use of the same information that is required in order to calculate a Laspeyres index (namely, base period expenditure shares, base period prices and current period prices), plus one other parameter (the elasticity of sub- stitution between commodities).
If information on this extra parameter can be obtained, then the resulting index can largely eliminate substitution bias and it can be calculated using basically the same information that is required to obtain the Laspeyres index.

17.8 The section starting with paragraph 17.65 con- siders the problem of deﬁning a true cost of living index when the consumer has annual preferences over com- modities but faces monthly (or quarterly) prices.
This section attempts to provide an economic foundation for the Lowe index studied in Chapter 15.

It also provides an introduction to the problems associated with the existence of seasonal commodities, which are considered at more length in Chapter 22.

The ﬁnal section deals 313 CONSUMER PRICE INDEX MANUAL: THEORY AND PRACTICE with situations where there may be a zero price for a commodity in one period, but where the price is non- zero in the other period.
The Konu¨ s cost of living index and observable bounds 17.9 This section deals with the theory of the cost of living index for a single consumer (or household) that was ﬁrst developed by the Russian economist, Konu¨ s (1924).

This theory relies on the assumption of optimizing behaviour on the part of economic agents (consumers or producers).
Thus, given a vector of com- modity prices pt that the household faces in a given time period t, it is assumed that the corresponding observed quantity vector qt is the solution to a cost minimization problem that consumer’s preference the or utility function f.1 Thus in contrast to the axiomatic the economic approach to index number approach does not assume that the two quantity vec- tors q0 and q1 are independent of the two price vectors p0 and p1.
In the economic approach, the period 0 quantity vector q0 is determined by the consumer’s preference function f and the period 0 vector of prices p0 that the consumer faces, and the period 1 quantity vector q1 is determined by the consumer’s preference function f and the period 1 vector of prices p1.
involves theory, 17.10 The economic approach to index number theory assumes that ‘‘the’’ consumer has well-deﬁned preferences over diﬀerent combinations of the n con- sumer commodities or items.2 Each combination of items can be represented by a positive quantity vector q:[q1, .

.

.

, qn].
.

.

, qn].

The consumer’s preferences over alter- native possible consumption vectors, q, are assumed to be representable by a continuous, non-decreasing and concave3 utility function f. Thus if f (q1)>f (q0), then the consumer prefers the consumption vector q1 to q0.

It is further assumed that the consumer minimizes the cost of achieving the period t utility level ut:f (qt) for periods t=0, 1.
Thus we assume that the observed period t consumption vector qt solves the following period t cost minimization problem: C(ut, pt) minq n P i=1 iqi : f (q)=ut f (qt) pt n =P i=1 iqt pt i for t=0, 1 (17:1) The period t price vector for the n commodities under consideration that the consumer faces is pt.
Note that the solution to the cost or expenditure minimization problem (17.1) for a general utility level u and general vector of commodity prices p deﬁnes the consumer’s cost function, C(u, p).

The cost function will be used below in order to deﬁne the consumer’s cost of living price index.

17.11 The Konu¨ s (1924) family of true cost of living indices pertaining to two periods where the consumer 1, .

.

.
.

.

, p0 faces the strictly positive price vectors p0 ( p0 n) and p1 ( p1 n) in periods 0 and 1, respectively, is deﬁned as the ratio of the minimum costs of achieving the same utility level u:f (q), where q:(q1, .

.

.

, qn) is a positive reference quantity vector: 1, .

.

.

, p1 PK ( p0, p1, q) C( f (q), p1) C( f (q), p0) (17:2) Note that deﬁnition (17.2) deﬁnes a family of price indices, because there is one such index for each refer- ence quantity vector q chosen.
17.12 It is natural to choose two speciﬁc reference quantity vectors q in deﬁnition (17.2): the observed base period quantity vector q0 and the current period quan- tity vector q1.
The ﬁrst of these two choices leads to the following Laspeyres–Konu¨s true cost of living index: PK ( p0, p1, q0) = = C( f (q0), p1) C( f (q0), p0) C( f (q0), p1) n P i=1 i q0 p0 i minq n P i=1 using (17:1) for t=0 i qi : f (q)=f (q0) p1 n P i=1 i q0 p0 i (17:3) using the deﬁnition of the cost minimization problem that deﬁnes C( f (q0), p1) n P i=1 n P i=1 i q0 p1 i i q0 p0 i since q0 (q0 problem 1, .

.

.
.

.

, q0 n) is feasible for the minimization 1 For a description of the economic theory of the input and output price indices, see Balk (1998a).

In the economic theory of the output price index, qt is assumed to be the solution to a revenue maximization problem involving the output price vector pt.
2 In this chapter, these preferences are assumed to be invariant over time, while in the following chapter, this assumption is relaxed (one of the environmental variables could be a time variable that shifts tastes).

3 Note that f is concave if and only if f (lq1+(1l)q2) lf (q1) + (1l) f (q2) for all 0 l 1 and all q1 0n and q2 0n.
Note also that q 0N means that each component of the N-dimensional vector q is non-negative, q 0n means that each component of q is positive and q>0n means that q 0n but q=0n; i.e., q is non-negative but at least one component is positive.

=PL( p0, p1, q0, q1) where PL is the Laspeyres price index.
Thus the (unob- servable) Laspeyres–Konu¨ s true cost of living index is bounded from above by the observable Laspeyres price index.4 17.13 The second of the two natural choices for a reference quantity vector q in deﬁnition (17.2) leads living to the following Paasche–Konu¨s true cost of 4 This inequality was ﬁrst obtained by Konu¨ s (1924; 1939, p. 17).

See also Pollak (1983).
314 THE ECONOMIC APPROACH TO INDEX NUMBER THEORY: THE SINGLE-HOUSEHOLD CASE Figure 17.1 The Laspeyres and Paasche bounds to the true cost of living index q2 q 0* q1 = (q 1 1,q 2 1) q 0 = (q 1 0,q 2 0) q 1* index: PK ( p0, p1, q1) = = C( f (q1), p1) C( f (q1), p0) n P i=1 i q1 p1 i C( f (q1), p0) n P i=1 n P i=1 using (17:1) for t=1 i q1 p1 i (17:4) minq p0 i qi : f (q)=f (q1) using the deﬁnition of the cost minimization problem that deﬁnes O A B C D E F q1 Cð f ðq0Þ; p0Þ n P i=1 n P i=1 i q1 p1 i p0 i q1 i since q1 (q1 1, .
.

.

, q1 n) is feasible for the minimization problem and thus n C( f (q1), p0) P i=1 i q1 p0 i and hence 1 C( f (q1), p0) 1 n P i=1 p0 i q1 i =PP( p0, p1, q0, q1) 2q0 1q0 17.14 1, q0 1+p0 where PP is the Paasche price index.

Thus the (unobser- vable) Paasche–Konu¨ s true cost of living index is boun- ded from below by the observable Paasche price index.5 It is possible to illustrate the two inequalities (17.3) and (17.4) if there are only two commodities; see Figure 17.1.
The solution to the period 0 cost minimization problem is the vector q0.

The straight line C represents the consumer’s period 0 budget con- the set of quantity points q1, q2 such that straint, 2.

The curved line through q0 is p0 2q2=p0 1q1+p0 the consumer’s period 0 indiﬀerence curve, the set of points q1, q2 such that f (q1, q2)=f (q0 2); i.e., it is the set of consumption vectors that give the same utility as the observed period 0 consumption vector q0.
The solution to the period 1 cost minimization problem is the vector q1.

The straight line D represents the consumer’s period 1 budget constraint, the set of quantity points q1, q2 such that p1 2q2=p1 2.

The curved line through q1 is the consumer’s period 1 indiﬀerence curve, the set of points q1, q2 such that f (q1, q2)= f (q1 2); i.e., it is the set of consumption vectors that give the same utility as the observed period 1 con- sumption vector q1.
The point q0* solves the hypothe- tical problem of minimizing the cost of achieving the base period utility level u0:f (q0) when facing the period 2).

Thus we have C[u0, p1]= 1 price vector p1= ( p1 p1 1 +p1 1q0* 2 and the dashed line A is the corresponding isocost line p1 2q2=C[u0, p1].

Note that the hypo- thetical cost line A is parallel to the actual period 1 cost 1q1+p1 1q1+p1 1+p1 1, p1 1, q1 2q0* 1q1 2q1 5 This inequality is attributable to Konu¨ s (1924; 1939, p. 19); see also Pollak (1983).
1q0 1+p0 1q0 1q0 1+p0 1+p1 line D. From equation (17.3), the Laspeyres–Konu¨ s true 2q0 index is C[u0, p1]=[ p0 2], while the ordinary Las- peyres index is [ p1 2q0 2]=[ p0 2q0 2].

Since the denominators for these two indices are the same, the diﬀerence between the indices is attributable to the dif- ferences in their numerators.

In Figure 17.1, this diﬀer- ence in the numerators is expressed by the fact that the cost line through A lies below the parallel cost line through B.
Now if the consumer’s indiﬀerence curve through the observed period 0 consumption vector q0 were L-shaped with vertex at q0, then the consumer would not change his or her consumption pattern in response to a change in the relative prices of the two commodities while keeping a ﬁxed standard of living.
In this case, the hypothetical vector q0* would coincide with q0, the dashed line through A would coincide with the dashed line through B and the true Laspeyres–Konu¨ s index would coincide with the ordinary Laspeyres index.

However, L-shaped indiﬀerence curves are not generally consistent with consumer behaviour; i.e., when the price of a commodity decreases, consumers generally demand more of it.

Thus, in the general case, there will be a gap between the points A and B.
The magnitude of this gap represents the amount of substitution bias between the true index and the corresponding Laspeyres index; i.e., the Laspeyres index will generally be greater than the corresponding true cost of living index, PK( p0, p1, q0).

1, p0 2).

Thus we have C[u1, p0]= p0 17.15 Figure 17.1 can also be used to illustrate the inequality (17.4).
First note that the dashed lines E and F are parallel line through to the period 0 isocost C. The point q1* solves the hypothetical problem of minimizing the cost of achieving the current period utility level u1:f (q1) when facing the period 0 price vector p0=( p0 2 and the dashed line E is the corresponding isocost line p1 1q1+p1 (17.4), the 2]=C[u1, p0], Paasche–Konu¨ s true index is [ p1 1+p1 1q1 2]= while the ordinary Paasche index is [ p0 2].
Since the numerators for these two indices are the same, the diﬀerence between the indices is attri- butable to the diﬀerences in their denominators.

In Figure 17.1, this diﬀerence in the denominators is expressed by the fact that the cost line through E lies below the parallel cost line through F. The magnitude of this diﬀerence 2q2=C[u0, p1].
From equation 1+ p1 1 +p0 2q1 [ p1 1+p0 2q1* 1q1* 1q1 2q1 1q1 2q1 315 CONSUMER PRICE INDEX MANUAL: THEORY AND PRACTICE represents the amount of substitution bias between the true index and the corresponding Paasche index; i.e., the Paasche index will generally be less than the correspond- ing true cost of living index, PK( p0, p1, q1).

Note that this inequality goes in the opposite direction to the previous inequality between the two Laspeyres indices.
The reason for this change in direction is attributable to the fact that one set of diﬀerences between the two indices takes place in the numerators of the indices (the Laspeyres inequal- ities), while the other set takes place in the denominators of the indices (the Paasche inequalities).
17.16 The bound (17.3) on the Laspeyres–Konu¨ s true cost of living index PK( p0, p1, q0) using the base period level of utility as the living standard is one-sided, as is the bound (17.4) on the Paasche–Konu¨ s true cost of living index PK( p0, p1, q1) using the current period level of utility as the living standard.
In a remarkable result, Konu¨ s (1924; 1939, p. 20) showed that there exists an intermediate consumption vector q* that is on the straight line joining the base period consumption vector q0 and the current period consumption vector q1 such that the corresponding (unobservable) true cost of living index PK ( p0, p1, q) is between the observable Laspeyres and Paasche indices, PL and PP.6 Thus we have the existence of a number l* between 0 and 1 such that PL PK ( p0, p1, l*q0+(1l*)q1) PP or PP PK ( p0, p1, l*q0+(1l*)q1) PL (17:5) The inequalities (17.5) are of some practical importance.
If the observable (in principle) Paasche and Laspeyres indices are not too far apart, then taking a symmetric average of these indices should provide a good approx- imation to a true cost of living index where the reference standard of living is somewhere between the base and current period living standards.
To determine the precise symmetric average of the Paasche and Laspeyres indices, appeal can be made to the results in paragraphs 15.18 to 15.32 in Chapter 15, and the geometric mean of the Paasche and Laspeyres indices can be justiﬁed as being the ‘‘best’’ average, which is the Fisher price index.

Thus the Fisher ideal price index receives a fairly strong jus- tiﬁcation as a good approximation to an unobservable theoretical cost of living index.
17.17 The bounds (17.3)–(17.5) are the best that can be obtained on true cost of living indices without mak- ing further assumptions.

Further assumptions are made below on the class of utility functions that describe the consumer’s tastes for the n commodities under con- sideration.

With these extra assumptions, the con- sumer’s true cost of living can be determined exactly.
The true cost of living index when preferences are homothetic 17.18 Up to now, the consumer’s preference func- tion f did not have to satisfy any particular homogeneity 6 For more recent applications of the Konu¨ s method of proof, see Diewert (1983a, p. 191) for an application to the consumer context and Diewert (1983b, pp.

1059–1061) for an application to the producer context.

316 assumption.
316 assumption.

For the remainder of this section, it is assumed that f is (positively) linearly homogeneous.7 In the economics literature, this is known as the assump- tion of homothetic preferences.8 This assumption is not strictly justiﬁed from the viewpoint of actual economic behaviour, but it leads to economic price indices that are living.9 independent of Under this assumption, the consumer’s expenditure or function, C(u, p) deﬁned by equation (17.1), cost decomposes as follows.
For positive commodity prices p 0N and a positive utility level u, then, using the deﬁnition of C as the minimum cost of achieving the given utility level u, the following equalities can be derived: the consumer’s standard of C(u, p) minq piqi : f (q1, .

.

.

, qn) u = minq piqi : n P i=1 n P i=1 dividing by u > 0 n P i=1 piqi : f 1 u = minq f (q1, .

.

.

, qn) 1 q1 u , .

.

.


, 1 qn u using the linear homogeneity of f =u minq =u minz n P i=1 n P i=1 piqi u : f q1 u , .

.

.

, 1 qn u pizi : f (z1, .

.

.

, zn) 1 letting =uC(1, p) using definition (17:1) zi= qi u =uc( p) (17:6) 7 The linear homogeneity property means that f satisﬁes the following condition: f (lq)=lf (q) for all l>0 and all q 0n.
This assumption is fairly restrictive in the consumer context.

It implies that each indif- ference curve is a radial projection of the unit utility indiﬀerence curve.

It also implies that all income elasticities of demand are unity, which is contradicted by empirical evidence.

8 More precisely, Shephard (1953) deﬁned a homothetic function to be a monotonic transformation of a linearly homogeneous function.
However, if a consumer’s utility function is homothetic, it can always be rescaled to be linearly homogeneous without changing consumer behaviour.

Hence, the homothetic preferences assumption can simply be identiﬁed with the linear homogeneity assumption.

9 This particular branch of the economic approach to index number theory is attributable to Shephard (1953; 1970) and Samuelson and Swamy (1974).
Shephard in particular realized the importance of the homotheticity assumption in conjunction with separability assump- tions in justifying the existence of sub-indices of the overall cost of living index.
It should be noted that, if the consumer’s change in real income or utility between the two periods under consideration is not too large, then assuming that the consumer has homothetic pre- ferences will lead to a true cost of living index which is very close to Laspeyres–Konu¨ s and Paasche–Konu¨ s true cost of living indices deﬁned by equations (17.3) and (17.4).
Another way of justifying the homothetic preferences assumption is to use equation (17.49), which justiﬁes the use of the superlative To¨ rnqvist–Theil index PT in the context of non-homothetic preferences.

Since PT is usually numeri- cally close to other superlative indices that are derived using the homothetic preferences assumption, it can be seen that the assumption of homotheticity will usually not be empirically misleading in the index number context.
THE ECONOMIC APPROACH TO INDEX NUMBER THEORY: THE SINGLE-HOUSEHOLD CASE where c( p):C(1, p) is the unit cost function that corre- sponds to f.10 It can be shown that the unit cost function c( p) satisﬁes the same regularity conditions that f satisﬁes; i.e., c( p) is positive, concave and (positively) linearly homogeneous for positive price vectors.11 Sub- stituting equation (17.6) into equation (17.1) and using ut=f (qt) leads to the following equation: equal to the value ratio) has the following form: Q( p0, p1, q0, q1) n P i=1 i q1 p1 i n P i=1 iqt pt iPK ( p0, p1, q) n P i=1 iqt pt i=c( pt)f (qt) for t=0, 1 (17:7) twice Thus, under the linear homogeneity assumption on the utility function f, observed period t expenditure on the n commodities is equal to the period t unit cost c( pt) of achieving one unit of utility times the period t utility level, f (qt).
Obviously, the period t unit cost, c( pt), can be identiﬁed as the period t price level Pt and the period t level of utility, f (qt), as the period t quantity level Qt.12 17.19 The linear homogeneity assumption on the consumer’s preference function f leads to a simpliﬁca- tion for the family of Konu¨ s true cost of living indices, PK( p0, p1, q), deﬁned by equation (17.2).
Using this deﬁnition for an arbitrary reference quantity vector q: PK ( p0, p1, q) = = C( f (q), p1) C( f (q), p0) c( p1)f (q) c( p0)f (q) c( p1) c( p0) using (17:6) twice (17:8) Thus under the homothetic preferences assumption, the entire family of Konu¨ s true cost of living indices collapses to a single index, c( p1)/c( p0), the ratio of the minimum costs of achieving unit utility level when the consumer faces period 1 and 0 prices respectively.
Put another way, under the homothetic preferences assumption, PK( p0, p1, q) is independent of the reference quantity vector q.
17.20 If the Konu¨ s true cost of living index deﬁned by the right-hand side of equation (17.8) is used as the price index concept, then the corresponding implicit quantity index deﬁned using the product test (i.e., the product of the price index times the quantity index is 10 Economists will recognize the producer theory counterpart to the result C(u, p)=uc( p): if a producer’s production function f is subject to constant returns to scale, then the corresponding total cost function C(u, p) is equal to the product of the output level u times the unit cost c( p).
11 Obviously, the utility function f determines the consumer’s cost function C(u, p) as the solution to the cost minimization problem in the ﬁrst line of equation (17.6).

Then the unit cost function c( p) is deﬁned as C(1, p).

Thus f determines c. But we can also use c to determine f under appropriate regularity conditions.

In the economics literature, this is known as duality theory.
For additional material on duality theory and the properties of f and c, see Samuelson (1953), Shephard (1953) and Diewert (1974a; 1993b, pp.107–123).

12 There is also a producer theory interpretation of the above theory; i.e., let f be the producer’s (constant returns to scale) production function, let p be a vector of input prices that the producer faces, let q be an input vector and let u=f (q) be the maximum output that can be produced using the input vector q.
C(u; p) minq f Pn i=1 piqi: f ðqÞ ug is the producer’s cost function in this case and c( pt) can be identiﬁed as the period t input price level, while f (qt) is the period t aggregate input level.
= c( p1)f (q1) c( p0)f (q0)PK ( p0, p1, q) using ð17:7Þ = = c( p1)f (q1) c( p0)f (q0) c( p1)=c( p0) g f f (q1) f (q0) using (17:8) (17:9) Thus, under the homothetic preferences assumption, the implicit quantity index that corresponds to the true cost of living price index c( p1)/c( p0) is the utility ratio f (q1)/ f (q0).

Since the utility function is assumed to be homo- geneous of degree one, this is the natural deﬁnition for a quantity index.
17.21 In subsequent material, two additional results from economic theory will be needed: Wold’s Identity and Shephard’s Lemma.

Wold’s (1944, pp.

69–71; 1953, p. 145) Identity is the following result.
Assuming that the consumer satisﬁes the cost minimization assumptions (17.1) for periods 0 and 1 and that the utility function f is diﬀerentiable at the observed quantity vectors q0 and q1, it can be shown13 that the following equation holds: = pt i n P k=1 kqt pt k @f (qt) @qi n P k=1 qt k @f (qt) @qk for t=0, 1 and k=1, .

.

.

, n (17:10) where @f (qt)/@qi denotes the partial derivative of the utility function f with respect to the ith quantity qi, evaluated at the period t quantity vector qt.
17.22 If the homothetic preferences assumption is made and it is assumed that the utility function is linearly homogeneous, then Wold’s Identity can be simpliﬁed into an equation that will prove to be very useful:14 = @f (qt)=@qi f (qt) pt i n P k=1 kqt pt k for t=0, 1 and k=1, .

.

.

, n (17:11) 13 To prove this, consider the ﬁrst-order necessary conditions for the strictly positive vector qt to solve the period t cost minimization pro- blem.
The conditions of Lagrange with respect to the vector of q variables are: pt=lt !f (qt), where lt is the optimal Lagrange multi- plier and !f (qt) is the vector of ﬁrst-order partial derivatives of f evaluated at qt.

Note that this system of equations is the price equals a constant times marginal utility equations that are familiar to econo- mists.

Now take the inner product of both sides of this equation with respect to the period t quantity vector qt and solve the resulting equation for lt.
Substitute this solution back into the vector equation pt=lt !f (qt) and equation (17.10) is obtained.

14 Diﬀerentiate both sides of the equation f (lq)=lf (q) with respect to l, and then evaluate the resulting equation at l=1.

The equation Pn i=1fi(q)qi=f (q) is obtained where fi(q):@f (q)/@qi.

317 CONSUMER PRICE INDEX MANUAL: THEORY AND PRACTICE 17.23 Shephard’s (1953, p. 11) Lemma is the fol- lowing result.

Consider the period t cost minimization problem deﬁned by equation (17.1).
If the cost function C(u, p) is diﬀerentiable with respect to the components of the price vector p, then the period t quantity vector qt is equal to the vector of ﬁrst-order partial derivatives of the cost function with respect to the components of p: Superlative indices: The Fisher ideal index 17.27 Suppose the consumer has the following uti- lity function: f (q1, .

.

.

, qn) s ﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃ n P aikqiqk, i=1 n P k=1 qt i= @C(ut, pt) @pi for i=1, .

.

.
.

.

, n and t=0, 1 (17:12) where aik=aki for all i and k (17:17) 17.24 To explain why equation (17.12) holds, con- sider the following argument.

Because it is assumed that the observed period t quantity vector qt solves the cost minimization problem deﬁned by C(ut, pt), then qt must be feasible for this problem so it must be the case that f (qt)=ut.
Thus, qt is a feasible solution for the following cost minimization problem where the general price vector p has replaced the speciﬁc period t price vector pt: C(ut, p) minq n P i=1 piqi : f (q1, .

.

.

, qn) ut n P i=1 piqt i (17:13) 1, .

.

.

, qt where the inequality follows from the fact that qt (qt n) is a feasible (but usually not optimal) solu- tion for the cost minimization problem in equation (17.13).
Now deﬁne for each strictly positive price vector p the function g( p) as follows: Diﬀerentiating f (q) deﬁned by equation (17.17) with respect to qi yields the following equation: fi(q)= 1 2 s n 2 P k=1 aikqk ﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃ n P ajkqjqk j=1 n P k=1 for i=1, .

.

.

, n n P k=1 aikqk f (q) = (17:18) where fi(q):@f (qt)/@qi.

In order to obtain the ﬁrst equation in (17.18), it is necessary to use the symmetry conditions, aik=aki.
Now evaluate the second equation in (17.18) at the observed period t quantity vector qt (qt n) and divide both sides of the resulting equation by f (qt).

The following equations are obtained: 1, .

.

.

, qt n g( p) P i=1 piqt iC(ut, p) (17:14) fi(qt) f (qt) = n P k=1 f aikqt k f (qt) g2 for t=0, 1 and i=1, .

.

.

, n (17:19) where, as usual, p:( p1, .

.

.

, pn).
.

.

, pn).

Using equations (17.13) and (17.1), it can be seen that g( p) is minimized (over all strictly positive price vectors p) at p=pt.

Thus the ﬁrst- order necessary conditions for minimizing a diﬀerenti- able function of n variables hold, which simplify to equation (17.12).

17.25 If the homothetic preferences assumption is made and it is assumed that the utility function is linearly homogeneous, then using equation (17.6), Shephard’s Lemma (17.12) becomes: i=ut @c( pt) qt @pi for i=1, .

.

.
.

.

, n and t=0, 1 (17:15) Combining equations (17.15) and (17.7), the following equation is obtained: Assume cost minimizing behaviour for the consumer in periods 0 and 1.

Since the utility function f deﬁned by equation (17.17) is linearly homogeneous and diﬀer- entiable, equation (17.11) will hold.
Now recall the deﬁnition of the Fisher ideal quantity index, QF, deﬁned earlier in Chapter 15: QF ( p0, p1, q0, q1)= v u u u u u t ﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃ n P p0 i q1 i i=1 n P k=1 kq0 p0 k v u u u u u t ﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃ n P i q1 p1 i i=1 n P k=1 kq0 p1 k = @c( pt) @pi c( pt) qt i n P k=1 kqt pt k for i=1, .

.

.

, n and t=0, 1 using equation (17:11) for t=0 17.26 Note the symmetry of equation (17.16) with equation (17.11).
It is these two equations that will be used in subsequent material in this chapter.
(17:16) 318 s ﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃ n P fi(q0) i=1 q1 i f (q0) = s ﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃ n P fi(q0) i=1 q1 i f (q0) = v u u u u u t ﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃ n P i q1 p1 i i=1 n P k=1 kq0 p1 k , v u u u u u t ﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃ n P kq0 p1 k k=1 n P i=1 i q1 p1 i THE ECONOMIC APPROACH TO INDEX NUMBER THEORY: THE SINGLE-HOUSEHOLD CASE s ﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃ n P fi(q0) i=1 q1 i f (q0) = , ﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃ s n P i=1 fi(q1) q0 i f (q1) using equation (17:11) for t=1 s = ﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃ n P aikq0 k i=1 q1 i f (q0) n P k=1 g2 f s , ﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃ aikq1 k n P i=1 n P k=1 q0 i f (q1) g2 f using equation (17:19) s s , ﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃ = ﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃ 1 f (q0) g2 f 1 f (q1) g2 f using equation (17:17) and cancelling terms = f (q1) f (q0) (17:20) Thus under the assumption that the consumer engages in cost-minimizing behaviour during periods 0 and 1 and has preferences over the n commodities that corre- spond to the utility function deﬁned by equation (17.17), the Fisher ideal quantity index QF is exactly equal to the true quantity index, f (q1Þ=f ðq0).15 17.28 As was noted in paragraphs 15.18 to 15.23 of Chapter 15, the price index that corresponds to the Fisher quantity index QF using the product test (15.3) is the Fisher price index PF, deﬁned by equation (15.12).
Let c( p) be the unit cost function that corresponds to the homogeneous quadratic utility function f deﬁned by equation (17.17).
Then using equations (17.16) and (17.20), it can be seen that PF ( p0, p1, q0, q1)= c( p1) c( p0) (17:21) Thus, under the assumption that the consumer engages in cost-minimizing behaviour during periods 0 and 1 and has preferences over the n commodities that corre- spond to the utility function deﬁned by equation (17.17), the Fisher ideal price index PF is exactly equal to the true price index, c( p1)/c( p0).

17.29 A twice continuously diﬀerentiable function f (q) of n variables q:(q1, .

.

.
.

.

, qn) can provide a second- order approximation to another such function f *(q) around the point q*, if the level and all the ﬁrst-order and second-order partial derivatives of the two func- tions coincide at q*.

It can be shown16 that the homo- geneous quadratic function f deﬁned by equation (17.17) can provide a second-order approximation to an arbi- trary f* around any (strictly positive) point q* in the linearly homogeneous functions.
Thus the class of homogeneous quadratic functional form deﬁned by equation (17.17) is a ﬂexible functional form.17 Diewert (1976, p. 117) formula Q( p0, p1, q0, q1) to the true that was exactly equal quantity index f (q1)/f (q0) (where f is a ﬂexible functional termed an index number 15 For the early history of this result, see Diewert (1976, p. 184).

16 See Diewert (1976, p. 130) and let the parameter r equal 2.

17 Diewert (1974a, p. 133) introduced this term into the economics literature.
form) a superlative index number formula.18 Equation (17.20) and the fact that the homogeneous quadratic function f deﬁned by equation (17.17) is a ﬂexible functional form show that the Fisher ideal quantity index QF deﬁned by equation (15.14) is a superlative index number formula.

Since the Fisher ideal price index PF satisﬁes equation (17.21), where c( p) is the unit cost function that is generated by the homogeneous quad- ratic utility function, PF is also called a superlative index number formula.
17.30 It is possible to show that the Fisher ideal price index is a superlative index number formula by a dif- ferent route.

Instead of starting with the assumption that the consumer’s utility function is the homogeneous it is quadratic function deﬁned by equation (17.17), possible to start with the assumption that the consumer’s unit cost function is a homogeneous quadratic.19 Thus, suppose that the consumer has the following unit cost function: c( p1, .

.

.
.

.

, pn) s ﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃ n P bik pi pk i=1 n P k=1 where bik=bki for all i and k: (17:22) Diﬀerentiating c( p) deﬁned by equation (17.22) with respect to pi yields the following equations: ci( p)= 1 2 s n 2 P k=1 bik pk ﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃ n P bjk pj pk j=1 n P k=1 for i=1, .

.

.

, n n P k=1 bik pk c(q) = (17:23) where ci( p):@c( pt)/@pi.

In order to obtain the ﬁrst equation in (17.23), it is necessary to use the symmetry conditions.
Now evaluate the second equation in (17.23) at the observed period t price vector pt ( pt 1, .

.

.

, pt n) and divide both sides of the resulting equation by c(pt).

The following equation is obtained: ci( pt) c( pt) = bikpt k n P k=1 f for t=0, 1 and i=1, .

.

.

, n (17:24) g2 c( pt) As cost-minimizing behaviour for the consumer in per- iods 0 and 1 is being assumed and, since the unit cost function c deﬁned by equation (17.22) is diﬀerentiable, equations (17.16) will hold.
Now recall the deﬁnition of 18 Fisher (1922, p. 247) used the term superlative to describe the Fisher ideal price index.

Thus, Diewert adopted Fisher’s terminology but attempted to give some precision to Fisher’s deﬁnition of super- lativeness.

Fisher deﬁned an index number formula to be superlative if it approximated the corresponding Fisher ideal results using his data set.
19 Given the consumer’s unit cost function c( p), Diewert (1974a, p. 112) showed that the corresponding utility function f (q) can be deﬁned as f (q) 1= maxp follows: for a strictly positive quantity vector q, fPn i=1piqi:c( p)=1g.
319 CONSUMER PRICE INDEX MANUAL: THEORY AND PRACTICE the Fisher ideal price index, PF, given by equation (15.12) in Chapter 15: PF ( p0, p1, q0, q1)= v u u u u u t ﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃ n P i q0 p1 i i=1 n P k=1 kq0 p0 k v u u u u u t s = ﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃ ci( p0) n P c( p0) i=1 p1 i p0 kq1 k ﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃ n P i q1 p1 i i=1 n P k=1 v u u u u u t ﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃ n P i q1 p1 i i=1 n P k=1 p0 kq1 k using equation (17:16) for t=0 s = s = ﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃ ci( p0) n P c( p0) i=1 p1 i ﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃ ci( p0) n P c( p0) i=1 p1 i , v u u u u u t ﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃ n P p0 kq1 k k=1 n P i=1 , ﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃ ci( p1) p0 i c( p1) i q1 p1 i n P i=1 s using equation (17:16) for t=1 s ﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃ 1 c( p0) = , ﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃ s 1 c( p1) f g2 g2 using equation (17:22) and cancelling terms c( p1) c( p0) = f : (17:25) Thus, under the assumption that the consumer engages in cost-minimizing behaviour during periods 0 and 1 and has preferences over the n commodities that corre- spond to the unit cost function deﬁned by equation (17.22), the Fisher ideal price index PF is exactly equal to the true price index, c( p1)/c( p0).20 17.31 Since the homogeneous quadratic unit cost function c( p) deﬁned by equation (17.22) is also a ﬂexible functional form, the fact that the Fisher ideal price index PF exactly equals the true price index c( p1)/c( p0) means that PF is a superlative index number formula.21 17.32 Suppose that the bik coeﬃcients in equation (17.22) satisfy the following restrictions:
 bik=bibk for i, k=1, .
.

.

, n (17:26) where the n numbers bi are non-negative.

In this special case of equation (17.22), it can be seen that the unit cost 20 This result was obtained by Diewert (1976, pp.

133–134).

21 Note that it has been shown that the Fisher index PF is exact for the preferences deﬁned by equation (17.17), as well as the preferences that are dual to the unit cost function deﬁned by equation (17.22).

These two classes of preferences do not coincide in general.
However, if the n by n symmetric matrix A of the aik has an inverse, then it can be shown that the n by n matrix B of the bik will equal A1.

320 function simpliﬁes as follows: c( p1, .

.

.

, pn) s = s n P k=1 ﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃ n P bibkpipk i=1 ﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃ n n P P bkpk i=1 k=1 bipi n =P i=1 bipi (17:27) Substituting equation (17.27) into Shephard’s Lemma (17.15) yields the following expressions for the period t quantity vectors, qt: i=ut @c( pt) qt @pi i=1, .
.

.

, n; t=0, 1 =biut (17:28) Thus if the consumer has the preferences that corre- spond to the unit cost function deﬁned by equation (17.22) where the bik satisfy the restrictions (17.26), then the period 0 and 1 quantity vectors are equal to a multiple of the vector b:(b1, .

.

.

, bn); i.e., q0= b u0 and q1=b u1.

Under these assumptions, the Fisher, Paasche and Laspeyres indices, PF, PP and PL, all coincide.
The preferences which correspond to the unit cost function deﬁned by equation (17.27) are, however, not consistent with normal consumer beha- viour since they imply that the consumer will not sub- stitute away from more expensive commodities to cheaper commodities if relative prices change going from period 0 to 1.
Quadratic mean of order r superlative indices 17.33 It turns out that there are many other super- i.e., there exist many lative index number formulae; quantity indices Q( p0, p1, q0, q1) that are exactly equal to f (q1)/f (q0) and many price indices P( p0, p1, q0, q1) that are exactly equal to c( p1)/c( p0), where the aggregator function f or the unit cost function c is a ﬂexible func- tional form.

Two families of superlative indices are deﬁned below.
17.34 Suppose the consumer has quadratic mean of order r utility function.22 the following f r(q1, .

.

.

, qn) s r ﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃ n P i qr=2 aikqr=2 i=1 n P k=1 k (17:29) where the parameters aik satisfy the symmetry condi- tions aik=aki for all i and k and the parameter r satisﬁes the restriction r=0.
Diewert (1976, p. 130) showed that the utility function f r deﬁned by equation (17.29) is a ﬂexible functional form; i.e., it can approximate an arbitrary twice linearly homogeneous functional form to the second order.

Note that when r=2, f r equals the homogeneous quadratic function deﬁned by equation (17.17).

continuously diﬀerentiable 22 The terminology is attributable to Diewert (1976, p. 129).
THE ECONOMIC APPROACH TO INDEX NUMBER THEORY: THE SINGLE-HOUSEHOLD CASE 17.35 Deﬁne the quadratic mean of order r quantity index Qr by: Qr( p0, p1, q0, q1) s r s r i =q0 ﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃ n P i )r=2 s0 i (q1 i=1 ﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃ n P i )r=2 i (q1 s1 i=1 i =q0 (17:30) i=Pn where st share for commodity i as usual.
i pt k=1pt kqt iqt k is the period t expenditure 17.36 Using exactly the same techniques as were used in paragraphs 17.27 to 17.32, it can be shown that Qr is exact for the aggregator function f r deﬁned by equation (17.29); i.e., the following exact relationship between the quantity index Qr and the utility function f r holds: Qr( p0, p1, q0, q1)= f r(q1) f r(q0) (17:31) Thus under the assumption that the consumer engages in cost-minimizing behaviour during periods 0 and 1 and has preferences over the n commodities that corre- spond to the utility function deﬁned by equation (17.29), the quadratic mean of order r quantity index QF is exactly equal to the true quantity index, f r(q1)/f r(q0).23 Since Qr is exact for f r and f r is a ﬂexible functional form, it can be seen that the quadratic mean of order r quantity index Qr is a superlative index for each r=0.
Thus there is an inﬁnite number of superlative quantity indices.

17.37 For each quantity index Qr, the product test (15.3) in Chapter 15 can be used in order to deﬁne the corresponding implicit quadratic mean of order r price index Pr*: Pr*( p0, p1, q0, q1) n P i=1 i q1 p1 i n P i=1 i q0 p0 i Qr( p0, p1, q0, q1) = cr*( p1) cr*( p0) (17:32) where cr* is the unit cost function that corresponds to the aggregator function f r deﬁned by equation (17.29).
For each r=0, the implicit quadratic mean of order r price index Pr* is also a superlative index.

17.38 When r=2, Qr deﬁned by equation (17.30) simpliﬁes to QF, the Fisher ideal quantity index, and Pr* deﬁned by equation (17.32) simpliﬁes to PF, the Fisher ideal price index.
When r=1, Qr deﬁned by equation (17.30) simpliﬁes to: Q1( p0, p1, q0, q1) s ﬃﬃﬃﬃﬃ q1 i q0 i s = ﬃﬃﬃﬃﬃ q0 i q1 i s0 i s1 i n P i=1 n P i=1 n P i=1 n P i=1 i q1 p1 i i q0 p0 i n P i=1 n P i=1 i q0 p0 i i q1 p1 i s s ﬃﬃﬃﬃﬃ q1 i q0 i ﬃﬃﬃﬃﬃ q0 i q1 i 23 See Diewert (1976, p. 130).
= = = n P i=1 n P i=1 n P i=1 n P i=1 n P i=1 n P i=1 i q1 p1 i i q0 p0 i n P i=1 n P i=1 p ﬃﬃﬃﬃﬃﬃﬃﬃﬃ q0 i q1 i p0 i p ﬃﬃﬃﬃﬃﬃﬃﬃﬃ q0 i q1 i p1 i i q1 p1 i i q0 p0 i n , P i=1 n P i=1 p ﬃﬃﬃﬃﬃﬃﬃﬃﬃ q0 i q1 i p1 i p ﬃﬃﬃﬃﬃﬃﬃﬃﬃ q0 i q1 i p0 i i q1 p1 i i q0 p0 i =PW ( p0, p1, q0, q1) (17:33) where PW is the Walsh price index deﬁned previously by equation (15.19) in Chapter 15.

Thus P1* is equal to PW, the Walsh price index, and hence it is also a superlative price index.
17.39 Suppose the consumer has the following quadratic mean of order r unit cost function:24 s r cr( p1, .

.

.

, pn) ﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃ n P i pr=2 bikpr=2 i=1 n P k=1 k (17:34) where the parameters bik satisfy the symmetry condi- tions bik=bki for all i and k, and the parameter r satisﬁes the restriction r=0.
Diewert (1976, p. 130) showed that the unit cost function cr deﬁned by equation (17.34) is a ﬂexible functional form; i.e., it can approximate an arbitrary twice linearly homogeneous functional form to the second order.

Note that when r=2, cr equals the homogeneous quadratic function deﬁned by equation (17.22).
continuously diﬀerentiable 17.40 Deﬁne the quadratic mean of order r price index Pr by: Pr( p0, p1, q0, q1) s r s r s0 i ﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃ r=2 p1 n P i p0 i=1 i ﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃ r=2 p1 n P i p0 i=1 i s1 i (17:35) i= Pn where st share for commodity i as usual.
k=1 pt i pt kqt iqt k is the period t expenditure 17.41 Using exactly the same techniques as were used in paragraphs 17.27 to 17.32, it can be shown that Pr is exact for the aggregator function deﬁned by equation (17.34); i.e., the following exact relationship between the index number formula Pr and the unit cost function cr holds: Pr( p0, p1, q0, q1)= cr( p1) cr( p0) (17:36) Thus, under the assumption that the consumer engages in cost-minimizing behaviour during periods 0 and 1, and has preferences over the n commodities that cor- respond to the unit cost function deﬁned by equation 24 This terminology is attributable to Diewert (1976, p. 130), this unit cost function being ﬁrst deﬁned by Denny (1974).
321 CONSUMER PRICE INDEX MANUAL: THEORY AND PRACTICE (17.34), the quadratic mean of order r price index PF is exactly equal to the true price index, cr( p1)/cr( p0).25 Since Pr is exact for cr and cr is a ﬂexible functional form, it can be seen that the quadratic mean of order r price index Pr is a superlative index for each r=0.

Thus there are an inﬁnite number of superlative price indices.
17.42 For each price index Pr, the product test (15.3) in Chapter 15 can be used in order to deﬁne the corre- sponding implicit quadratic mean of order r quantity index Qr*: Qr*( p0, p1, q0, q1) n P i=1 i q1 p1 i n P i=1 i q0 p0 i Pr( p0, p1, q0, q1) = f r*( p1) f r*( p0) (17:37) where f r* is the aggregator function that corresponds function cr deﬁned by equation to the unit cost (17.34).26 For each r=0, the implicit quadratic mean of order r quantity index Qr* is also a superlative index.
17.43 When r=2, Pr deﬁned by equation (17.35) simpliﬁes to PF, the Fisher ideal price index, and Qr* deﬁned by equation (17.37) simpliﬁes to QF, the Fisher ideal quantity index.
When r=1, Pr deﬁned by equation (17.35) simpliﬁes to: P1( p0, p1, q0, q1) = s ﬃﬃﬃﬃﬃ p1 i p0 i s = ﬃﬃﬃﬃﬃ p0 i p1 i n P i=1 n P i=1 s0 i s1 i i q1 p1 i i q0 p0 i n P i=1 n P i=1 i q0 p0 i i q1 p1 i s s ﬃﬃﬃﬃﬃ p1 i p0 i ﬃﬃﬃﬃﬃ p0 i p1 i i q1 p1 i i q0 p0 i n P i=1 n P i=1 p ﬃﬃﬃﬃﬃﬃﬃﬃﬃ p0 i p1 i q0 i p ﬃﬃﬃﬃﬃﬃﬃﬃﬃ p0 i p1 i q1 i n P i=1 n P i=1 n P i=1 n P i=1 = n P i=1 n P i=1 i q1 p1 i i q0 p0 i n , P i=1 n P i=1 p ﬃﬃﬃﬃﬃﬃﬃﬃﬃ p0 i p1 i q1 i p ﬃﬃﬃﬃﬃﬃﬃﬃﬃ p0 i p1 i q0 i Superlative indices: The To¨ rnqvist index 17.44 In this section, the same assumptions that were made on the consumer in paragraphs 17.9 to 17.17 are made here.
In particular, it is not assumed that the consumer’s utility function f is necessarily linearly homogeneous as in paragraphs 17.18 to 17.43.

17.45 Before the main result is derived, a pre- liminary result is required.

Suppose the function of n variables, f (z1, .

.

.

, zn):f (z), is quadratic; i.e., f (z1, .

.

.

, zn) a0+P n aizi+ i=1 1 2 n P i=1 n P k=1 aikzizk and aik=aki for all i and k (17:39) where the ai and the aik are constants.
Let fi(z) denote the ﬁrst-order partial derivative of f evaluated at z with respect to the ith component of z, zi.

Let fik(z) denote the second-order partial derivative of f with respect to zi and zk.
Then it is well known that the sec- ond-order Taylor series approximation to a quadratic function is exact; i.e., if f is deﬁned by equation (17.39), then for any two points, z0 and z1, the following equa- tion holds: fi(z0) z1 i z0 i n f (z1)f (z0)=P i=1 1 2 + n P i=1 n P k=1 fik(z0) z1 i z0 i z1 kz0 k (17:40) It is less well known that an average of two ﬁrst-order Taylor series approximations to a quadratic function is also exact; i.e., if f is deﬁned by equation (17.39) above, then for any two points, z0 and z1, the following equa- tion holds:27 f (z1)f (z0)= 1 2 n P i=1 fi(z0)+fi(z1) z1 i z0 i (17:41) Diewert (1976, p. 118) and Lau (1979) showed that equation (17.41) characterized a quadratic function and called the equation the quadratic approximation lemma.
In this chapter, equation (17.41) will be called the quadratic identity.

17.46 Suppose func- the tion28 C(u, p), has the following translog functional form:29 consumer’s that cost = n P i=1 n P i=1 i q1 p1 i i q0 p0 i =QW ( p0, p1, q0, q1) (17:38) ln C(u, p) a0+P n ai ln pi+ i=1 n P k=1 aik ln pi ln pk+b0 ln u n 1 P 2 i=1 1 2 n +P i=1 bi ln pi ln u+ b00( ln u)2 (17:42) where QW is the Walsh quantity index deﬁned pre- viously in footnote 30 of Chapter 15.
Thus Q1* is equal to QW, the Walsh quantity index, and hence it is also a superlative quantity index.

25 See Diewert (1976, pp.

133–134).

26 The function f r* can be deﬁned by using cr as follows: f r*(q) 1= maxp f Pn i=1 piqi:cr( p)=1g.

322 where ln is the natural parameters ai, aik, and bi logarithm function and the following satisfy the 27 The proof of this and the foregoing relation is by straightforward veriﬁcation.

28 The consumer’s cost function was deﬁned by equation (17.6) above.
29 Christensen, Jorgenson and Lau (1971) introduced this function into the economics literature.

THE ECONOMIC APPROACH TO INDEX NUMBER THEORY: THE SINGLE-HOUSEHOLD CASE restrictions: n aik=aki, P n ai=1, P bi=0 and P n aik=0 i=1 i=1 k=1 for i, k=1, .

.

.

, n (17:43) These parameter restrictions ensure that C(u, p) deﬁned by equation (17.42) is linearly homogeneous in p, a property that a cost function must have.
It can be shown that the translog cost function deﬁned by equation (17.42) can provide a second-order Taylor series approx- imation to an arbitrary cost function.30 17.47 Assume that the consumer has preferences that correspond to the translog cost function and that the consumer engages in cost-minimizing behaviour during periods 0 and 1.

Let p0 and p1 be the period 0 and 1 observed price vectors, and let q0 and q1 be the period 0 and 1 observed quantity vectors.
These assumptions imply: n C(u0, p0)=P i=1 i q0 p0 i and C(u1, p1)=P i q1 p1 i (17:44) n i=1 where C is the translog cost function deﬁned above.
Now apply Shephard’s Lemma, equation (17.12), and the following equation results: the following equation is obtained: ln C(u*, p1) ln C(u*, p0) = = 1 2 1 2 n P i=1 n P i=1 @ ln C(u*, p0) @ ln pi + @ ln C(u*, p1) @ ln pi ln p1 i ln p0 i n ai+ P k=1 aik ln p0 k+bi ln u* +ai aik ln p1 k+bi ln u* ln p1 i ln p0 i n ai+ P k=1 aik ln p0 k+bi ln p ﬃﬃﬃﬃﬃﬃﬃﬃﬃ u0u1 +ai aik ln p1 k+bi ln p ﬃﬃﬃﬃﬃﬃﬃﬃﬃ u0u1 ln p1 i ln p0 i n ai+ P k=1 aik ln p0 k+bi ln u0+ai aik ln p1 k+bi ln u1 ln p1 i ln p0 i n + P k=1 1 2 n P i=1 n + P k=1 1 2 n P i=1 n + P k=1 1 2 n P i=1 = = = @ ln C(u0, p0) @ ln pi + @ ln C(u1, p1) @ ln pi ln p1 i ln p0 i for i=1, .
.

.

, n and t=0, 1 = 1 2 n P i=1 i +s1 s0 i ln p1 i ln p0 i using equation (17:47): qt i= = @C(ut, pt) @pi C(ut, pt) pt i @ ln C(ut, pt) @ ln pi Now use equation (17.44) to replace C(ut, pt) in equation (17.45).

After some cross multiplication, this becomes the following: iqt pt i kqt pt k st i= @ ln C(ut, pt) @ ln pi n P k=1 for i=1, .

.

.

, n and t=0, 1 or n i=ai+ P st k=1 aik ln pt k+bi ln ut for i=1, .

.

.
.

.

, n and t=0, 1 (17:47) where st i is the period t expenditure share on commodity i.

17.48 Deﬁne the geometric average of the period 0 and 1 utility levels as u*; i.e., deﬁne p ﬃﬃﬃﬃﬃﬃﬃﬃﬃ u0u1 u* (17:48) Now observe that the right-hand side of the equation that deﬁnes the natural logarithm of the translog cost function, equation (17.42), is a quadratic function of the variables zi:lnpi if utility is held constant at the level u*.
Hence the quadratic identity (17.41) can be applied, and 30 It can also be shown that, if all the bi=0 and b00=0, then C(u, p)=uC(1, p):uc( p); i.e., with these additional restrictions on the parameters of the general translog cost function, homothetic pref- erences are the result of these restrictions.

Note that it is also assumed that utility u is scaled so that u is always positive.
(17:45) (17:49) The last equation in (17.49) can be recognized as the logarithm of the To¨ rnqvist–Theil index number formula PT, deﬁned earlier by equation (15.81) in Chapter 15.
Hence, exponentiating both sides of equation (17.49) yields the following equality between the true cost of living between periods 0 and 1, evaluated at the intermediate utility level u* and the observable To¨ rnqvist–Theil index PT:31 (17:46) C(u*, p1) Cðu*, p0) =PT ( p0, p1, q0, q1) (17:50) Since the translog cost function which appears on the left-hand side of equation (17.49) is a ﬂexible functional form, the To¨ rnqvist–Theil price index PT is also a superlative index.
17.49 It is somewhat mysterious how a ratio of unobservable cost functions of the form appearing on the left-hand side of the above equation can be exactly estimated by an observable index number formula.

The key to this mystery is the assumption of cost-minimizing behaviour and the quadratic identity (17.41), along with the fact that derivatives of cost functions are equal to quantities, as speciﬁed by Shephard’s Lemma.
In fact, all the exact index number results derived in paragraphs 17.27 to 17.43 can be derived using trans- formations of the quadratic identity along with Shep- hard’s Lemma (or Wold’s Identity).32 Fortunately, the for most empirical applications, assuming that 31 This result is attributable to Diewert (1976, p. 122).

32 See Diewert (2002a).
323 CONSUMER PRICE INDEX MANUAL: THEORY AND PRACTICE consumer has (transformed) quadratic preferences will be an adequate assumption, so the results presented in paragraphs 17.27 to 17.49 are quite useful to index number practitioners who are willing to adopt the eco- nomic approach to index number theory.33 Essentially, the economic approach to index number theory provides a strong justiﬁcation for the use of the Fisher price index PF deﬁned by equation (15.12), the To¨ rnqvist–Theil price index PT deﬁned by equation (15.81), the implicit quadratic mean of order r price indices Pr* deﬁned by equation (17.32) (when r=1, this index is the Walsh price index deﬁned by equation (15.19) in Chapter 15) and the quadratic mean of order r price indices Pr deﬁned by equation (17.35).
In the next section, we ask if it matters which one of these formulae is chosen as ‘‘best’’.

The approximation properties of superlative indices 17.50 The results of paragraphs 17.27 to 17.49 pro- vide price statisticians with a large number of index number formulae which appear to be equally good from the viewpoint of the economic approach to index num- ber theory.

Two questions arise as a consequence of these results: Does it matter which of these formulae is chosen?
If it does matter, which formula should be chosen?

17.51 With respect to the ﬁrst question, Diewert (1978, p. 888) showed that all of the superlative index number formulae listed in paragraphs 17.27 to 17.49 approximate each other to the second order around any point where the two price vectors, p0 and p1, are equal and where the two quantity vectors, q0 and q1, are equal.
In particular, this means that the following equalities are valid for all r and s not equal to 0, provided that p0=p1 and q0=q1.34 PT ( p0, p1, q0, q1)=Pr( p0, p1, q0, q1)=Ps*( p0, p1, q0, q1) (17:51) @PT ( p0, p1, q0, q1) @pt i = @Pr( p0, p1, q0, q1) @pt i @Ps*( p0, p1, q0, q1) @pt i = for i=1, .

.

.
.

.

, n and t=0, 1 (17:52) 33 If, however, consumer preferences are non-homothetic and the change in utility is substantial between the two situations being com- pared, then it may be desirable to compute separately the Laspeyres– Konu¨ s and Paasche–Konu¨ s true cost of living indices deﬁned by equations (17.3) and (17.4), C(u0, p1)/C(u0, p0) and C(u1, p1)/C(u1, p0), respectively.
In order to do this, it would be necessary to use econo- metrics and estimate empirically the consumer’s cost or expenditure function.

34 To prove the equalities in equations (17.51) to (17.56), simply dif- ferentiate the various index number formulae and evaluate the deri- vatives at p0=p1 and q0=q1.
Actually, equations (17.51) to (17.56) are still true provided that p1=lp0 and q1=mq0 for any numbers l>0 and m>0; i.e., provided that the period 1 price vector is proportional to the period 0 price vector and that the period 1 quantity vector is pro- portional to the period 0 quantity vector.

324 @PT ( p0, p1, q0, q1) @qt i = = @Pr( p0, p1, q0, q1) @qt i @Ps*( p0, p1, q0, q1) @qt i for i=1, .

.

.
.

.

, n and t=0, 1 (17:53) @2PT ( p0, p1, q0, q1) i@pt k @pt = = @pt @2Pr( p0, p1, q0, q1) i@pt k @2Ps*( p0, p1, q0, q1) i@pt k @pt for i, k=1, .

.

.

, n and t=0, 1 (17:54) @2PT ( p0, p1, q0, q1) i@qt k @pt = @pt @2Pr( p0, p1, q0, q1) i@qt k @2Ps*( p0, p1, q0, q1) i@qt k = @pt for i, k=1, .

.

.

, n and t=0, 1 (17:55) @2PT ( p0, p1, q0, q1) i@qt k @qt = = @qt @2Pr( p0, p1, q0, q1) i@qt k @2Ps*( p0, p1, q0, q1) i@qt k @qt for i, k=1, .

.

.
.

.

, n for t=0, 1 (17:56) where the To¨ rnqvist–Theil price index PT is deﬁned by equation (15.81), the implicit quadratic mean of order r price indices P s* is deﬁned by equation (17.32) and the quadratic mean of order r price indices P r is deﬁned by equation (17.35).

Using the results in the previous paragraph, Diewert (1978, p. 884) concluded that ‘‘all superlative indices closely approximate each other’’.
17.52 The above conclusion is, however, not true even though the equations (17.51) to (17.56) are true.

The problem is that the quadratic mean of order r price indices P r and the implicit quadratic mean of order s price indices P s* are (continuous) functions of the para- meters r and s respectively.

Hence, as r and s become very large in magnitude, the indices P r and P s* can diﬀer substantially from, say, P2=PF, the Fisher ideal index.
In fact, using deﬁnition (17.35) and the limiting properties of means of order r,35 Robert Hill (2002, p. 7) showed that Pr has the following limit as r approaches plus or minus inﬁnity: lim r!+1 Pr( p0, p1, q0, q1)= lim r!

1 s Pr( p0, p1, q0, q1) = ﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃ p1 i mini p0 i p1 i p0 i maxi (17:57) 35 See Hardy, Littlewood and Po´ lya (1934).
THE ECONOMIC APPROACH TO INDEX NUMBER THEORY: THE SINGLE-HOUSEHOLD CASE Using Hill’s method of analysis, it can be shown that the implicit quadratic mean of order r price index has the following limit as r approaches plus or minus inﬁnity: lim r!+1 Pr*( p0, p1, q0, q1)= lim r!
1 Pr*( p0, p1, q0, q1, ) n P i=1 i q1 p1 i ﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃ p1 i mini p0 i p1 i p0 i maxi = s n P i=1 i q0 p0 i (17:58) Thus for r large in magnitude, P r and P r* can diﬀer substantially from PT, P1, P1*=PW (the Walsh price index) and P2=P2*=PF (the Fisher ideal index).36 17.53 Although Hill’s theoretical and empirical results demonstrate conclusively that not all superlative indices will necessarily closely approximate each other, there is still the question of how well the more commonly used superlative indices will approximate each other.
All the commonly used superlative indices, P r and P r*, fall into the interval 0 r 2.37 Hill (2002, p. 16) summar- ized how far apart the To¨ rnqvist and Fisher indices were, making all possible bilateral comparisons between any two data points for his time series data set as follows: The superlative spread S(0, 2) is also of interest since, in practice, To¨ rnqvist (r=0) and Fisher (r=2) are by far the two most widely used superlative indexes.
In all 153 bilateral comparisons, S(0, 2) is less than the Paasche– Laspeyres spread and on average, the superlative spread is only 0.1 per cent.

It is because attention, until now, has focussed almost exclusively on superlative indexes in the range 0 r 2 that a general misperception has persisted in the index number literature that all superlative indexes approximate each other closely.
Thus, for Hill’s time series data set covering 64 com- ponents of United States gross domestic product from 1977 to 1994 and making all possible bilateral compar- isons between any two years, the Fisher and To¨ rnqvist price indices diﬀered by only 0.1 per cent on average.

This close correspondence is consistent with the results of other empirical studies using annual time series data.38 Additional evidence on this topic may be found in Chapter 19.
17.54 In the earlier chapters of this manual, it is found that several index number formulae seem ‘‘best’’ when viewed from various perspectives.

Thus the Fisher ideal index PF=P2=P2* deﬁned by equation (15.12) seemed to be best from one axiomatic viewpoint, the To¨ rnqvist–Theil price index PT deﬁned by equation (15.81) seems to be best from another axiomatic per- 36 Hill (2002) documents this for two data sets.
His time series data consist of annual expenditure and quantity data for 64 components of United States gross domestic product from 1977 to 1994.

For this data set, Hill (2002, p. 16) found that ‘‘superlative indexes can diﬀer by more than a factor of two (i.e., by more than 100 per cent), even though Fisher and To¨ rnqvist never diﬀer by more than 0.6 per cent’’.

37 Diewert (1980, p. 451) showed that the To¨ rnqvist index PT is a limiting case of P r, as r tends to 0.
38 See, for example, Diewert (1978, p. 894) or Fisher (1922), which is reproduced in Diewert (1976, p. 135).

spective, as well as from the stochastic viewpoint, and the Walsh index PW deﬁned by equation (15.19) (which is equal to the implicit quadratic mean of order r price indices Pr* deﬁned by equation (17.32) when r=1) seems to be best from the viewpoint of the ‘‘pure’’ price index.
The results presented in this section indicate that for ‘‘normal’’ time series data, these three indices will give virtually the same answer.

To determine precisely which one of these three indices to use as a theoretical target or actual index, the statistical agency will have to decide which approach to bilateral index number theory is most consistent with its goals.
For most practical purposes, however, it will not matter which of these three indices is chosen as a theoretical target index for making price comparisons between two periods.

Superlative indices and two-stage aggregation 17.55 Most statistical agencies use the Laspeyres formula to aggregate prices in two stages.
At the ﬁrst stage of aggregation, the Laspeyres formula is used to aggregate components of the overall index (e.g., food, clothing, services); then at the second stage of aggrega- tion, these component sub-indices are further combined into the overall index.

The following question then naturally arises: does the index computed in two stages coincide with the index computed in a single stage?
Initially, this question is addressed in the context of the Laspeyres formula.39 17.56 Suppose that the price and quantity data for period t, pt and qt, can be written in terms of M sub- vectors as follows: pt=( pt1 , pt2 , .

.

.

, ptM ) and qt=(qt1, qt2 , .

.

.

, qtM ) for t=0, 1 (17:59) where the dimensionality of the subvectors ptm and qtm is Nm for m=1, 2, .

.

.
.

.

, M with the sum of the dimensions Nm equal to n. These subvectors correspond to the price and quantity data for subcomponents of the consumer price index for period t. Now construct sub-indices for each of these components going from period 0 to 1.

For the base period, set the price for each of these sub- components, say P 0 m for m=1, 2, .

.

.

M, equal to 1 and set the corresponding base period subcomponent quantities, say Q0 m for m=1, 2, .

.

.
.

.

, M, equal to the base period value for m= of consumption for 1, 2, .

.

.

, M: subcomponent that m 1 and Q0 P0 m P Nm i=1 i q0m p0m i for m=1, 2, .

.

.

, M (17:60) Now use the Laspeyres formula in order to construct a period 1 price for each subcomponent, say P1 m for m=1, 2, .

.

.

, M, of the CPI.

Since the dimensionality of 39 Much of the material in this section is adapted from Diewert (1978) and Alterman, Diewert and Feenstra (1999).
See also Balk (1996b) for a discussion of alternative deﬁnitions for the two-stage aggregation concept and references to the literature on this topic.

325 CONSUMER PRICE INDEX MANUAL: THEORY AND PRACTICE the subcomponent vectors, ptm and qtm , diﬀers from the dimensionality of the complete period t vectors of prices and quantities, pt and qt, it is necessary to use diﬀerent symbols for these subcomponent Laspeyres indices, say Pm L for m=1, 2, .

.

.
.

.

, M. Thus the period 1 subcomponent prices are deﬁned as follows: m Pm P1 L ( p0m, p1m, q0m, q1m) Nm P i=1 Nm P i=1 i q0m p1m i i q0m p0m i for m=1, 2, .

.

.

, M (17:61) Once the period 1 prices for the M sub-indices have been deﬁned by equation (17.61), then corresponding sub- component period 1 quantities Q1 m for m=1, 2, .

.

.
.

.

, M can be deﬁned by deﬂating the period 1 subcomponent values PNm i by the prices P1 m i q1m i=1p1m Nm P i=1 i q1m p1m i P1 m Q1 m for m=1, 2, .

.

.

, M (17:62) Now deﬁne subcomponent price and quantity vectors for each period t=0,1 using equations (17.60) to (17.62).

Thus deﬁne the period 0 and 1 subcomponent price vectors P0 and P1 as follows: P0=(P0 1, P0 2, .

.

.

, P0 M) 1M and P1=(P1 1, P1 2, .

.

.


, P1 M) (17:63) where 1M denotes a vector of ones of dimension M and the components of P1 are deﬁned by equation (17.61).

The period 0 and 1 subcomponent quantity vectors Q0 and Q1 are deﬁned as follows: 1, Q1 1, Q0 Q0=(Q0 2, .

.

.

, Q1 2, .

.

.

, Q0 M) and Q1=(Q1 M) (17:64) where the components of Q0 are deﬁned in equation (17.60) and the components of Q1 are deﬁned by equa- tion (17.62).
The price and quantity vectors in equations (17.63) and (17.64) represent the results of the ﬁrst-stage aggregation.

Now use these vectors as inputs into the second-stage aggregation problem; i.e., apply the Laspeyres price index formula, using the information in equations (17.63) and (17.64) as inputs into the index number formula.
Since the price and quantity vectors that are inputs into this second-stage aggregation pro- blem have dimension M instead of the single-stage for- mula which utilized vectors of dimension n, a diﬀerent symbol is required for the new Laspeyres index: this is chosen to be P* L. Thus the Laspeyres price index com- puted in two stages can be denoted as P* L(P0, P1, Q0, Q1).
Now ask whether this two-stage Laspeyres index equals the corresponding single-stage index PL that was studied in the previous sections of this chapter; i.e., ask whether L(P0, P1, Q0, Q1)=PL( p0, p1, q0, q1): P* (17:65) If the Laspeyres formula is used at each stage of each aggregation, the answer to the above question is yes: straightforward calculations show that the Laspeyres 326 index calculated in two stages equals the Laspeyres index calculated in one stage.
17.57 Now suppose that the Fisher or To¨ rnqvist formula is used at each stage of the aggregation.

That is, in equation (17.61), suppose that the Laspeyres formula L ( p0m, p1m, q0m, q1m) is replaced by the Fisher formula Pm Pm F ( p0m, p1m, q0m, q1m) or by the To¨ rnqvist formula Pm T ( p0m, p1m, q0m, q1m); and in equation (17.65), suppose that P* T ) and PL (p0, p1, q0, q1) is replaced by PF (or by PT).
Then is it the case that counterparts are obtained to the two- stage aggregation result for the Laspeyres formula, equation (17.65)?
The answer is no; it can be shown that, in general, L(P0, P1, Q0, Q1) is replaced by P* F (or by P* P* F (P0, P1, Q0, Q1) 6¼ PF ( p0, p1, q0, q1) and T (P0, P1, Q0, Q1) 6¼ PT ( p0, p1, q0, q1) P* (17:66) Similarly, it can be shown that the quadratic mean of order r index number formula Pr deﬁned by equation (17.35) and the implicit quadratic mean of order r index number formula Pr* deﬁned by equation (17.32) are also not consistent in aggregation.
formulae are not exactly consistent 17.58 Nevertheless, even though the Fisher and To¨ rnqvist in aggregation, it can be shown that these formulae are approximately consistent in aggregation.
More speciﬁ- cally, it can be shown that the two-stage Fisher formula P* F and the single-stage Fisher formula PF in the in- equality (17.66), both regarded as functions of the 4n variables in the vectors p0, p1, q0, q1, approximate each other to the second order around a point where the two price vectors are equal (so that p0=p1) and where the two quantity vectors are equal (so that q0=q1), and a similar result holds for the two-stage and single-stage To¨ rnqvist indices in equation (17.66).40 As was seen in the previous single-stage Fisher and the To¨ rnqvist indices have a similar approximation prop- erty, so all four indices in the inequality (17.66) approx- imate each other to the second order around an equal (or proportional) price and quantity point.
Thus for time series data, single-stage and two-stage normal Fisher and To¨ rnqvist indices will usually be numerically very close.
This result is illustrated in Chapter 19 for an artiﬁcial data set.41 section, 17.59 Similar approximate consistency in aggrega- tion results (to the results for the Fisher and To¨ rnqvist formulae explained in the previous paragraph) can be derived for the quadratic mean of order r indices, Pr, and for the implicit quadratic mean of order r indices, Pr*; see Diewert (1978, p. 889).

Nevertheless, the results the second-order of Hill (2002) again imply that 40 See Diewert (1978, p. 889).
In other words, a string of equalities similar to equations (17.51) to (17.56) holds between the two-stage indices and their single-stage counterparts.

In fact, these equalities are still true provided that p1=l p0 and q1=mq0 for any numbers l>0 and m>0.

41 For an empirical comparison of the four indices, see Diewert (1978, pp.

894–895).
894–895).

For the Canadian consumer data considered there, the chained two-stage Fisher in 1971 was 2.3228 and the corresponding chained two-stage To¨ rnqvist was 2.3230, the same values as for the corresponding single-stage indices.

THE ECONOMIC APPROACH TO INDEX NUMBER THEORY: THE SINGLE-HOUSEHOLD CASE approximation property of the single-stage quadratic mean of order r index Pr to its two-stage counterpart will break down as r approaches either plus or minus inﬁnity.
To see this, consider a simple example where there are only four commodities in total.

Let the ﬁrst 1=p0 price ratio p1 1 be equal to the positive number a, let the second two price ratios p1 i equal b and let the last price ratio p1 4=p0 4 equal c, where we assume a<c and a b c. Using Hill’s result (17.57), the limiting value of the single-stage index is: i =p0 lim r!+1 Pr( p0, p1, q0, q1)= lim r!
1 s Pr( p0, p1, q0, q1) = ﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃ p1 i mini p0 i p1 i p0 i maxi p = ﬃﬃﬃﬃﬃac (17:67) Now aggregate commodities 1 and 2 into a sub-aggre- gate and commodities 3 and 4 into another sub-aggre- gate.

Using Hill’s result (17.57) again, it is found that the limiting price index for the ﬁrst sub-aggregate is [ab]1/2 and the limiting price index for the second sub-aggre- gate is [bc]1/2.
Now apply the second stage of aggrega- tion and use Hill’s result once again to conclude that the limiting value of the two-stage aggregation using Pr as the index number formula is [ab2c]1/4.

Thus the lim- iting value as r tends to plus or minus inﬁnity of the single-stage aggregate over the two-stage aggregate is [ac]1=2=[ab2c]1=4=[ac= b2]1=4.
Now b can take on any value between a and c, and so the ratio of the single- stage limiting Pr to its two-stage counterpart can take on any value between [c/a]1/4 and [a/c]1/4.
Since c/a is less than 1 and a/c is greater than 1, it can be seen that the ratio of the single-stage to the two-stage index can be arbitrarily far from 1 as r becomes large in magnitude with an appropriate choice of the numbers a, b and c. 17.60 The results in the previous paragraph show that some caution is required in assuming that all superlative indices will be approximately consistent in aggregation.
However, for the three most commonly used superlative indices (the Fisher ideal PF, the To¨ rnqvist– Theil PT and the Walsh PW), the available empirical evidence indicates that these indices satisfy the con- sistency in aggregation property to a suﬃciently high degree of approximation that users will not be unduly troubled by any inconsistencies.42 The Lloyd–Moulton index number formula 17.61 The index number formula that will be dis- cussed in this section on the single-household economic approach to index number theory is a potentially very useful one for statistical agencies that are faced with the problem of producing a CPI in a timely manner.
The Lloyd–Moulton formula that will be discussed in this section makes use of the same information that is required in order to implement a Laspeyres index except for one additional piece of information.

42 See Chapter 19 for some additional evidence on this topic.

17.62 In this section, the same assumptions about the consumer are made that were made in paragraphs 17.18 to 17.26 above.
In particular, it is assumed that the consumer’s utility function f (q) linearly homo- geneous43 and the corresponding unit cost function is c( p).

It is supposed that the unit cost function has the following functional form: is c( p) a0 1=(1s) n P i=1 aip1s i if s 6¼ 1 or ln c( p) a0+P n i=1 ai ln pi if s=1 (17:68) where the ai and s are non-negative parameters with Pn i=1ai=1.
The unit cost function deﬁned by equation (17.68) corresponds to a constant elasticity of substitu- tion (CES) aggregator function, which was introduced into the economics literature by Arrow, Chenery, Minhas and Solow (1961).44 The parameter s is the elasticity of substitution; when s=0, the unit cost function deﬁned by equation (17.68) becomes linear in prices and hence corresponds to a ﬁxed coeﬃcients aggregator function which exhibits zero substitutability between all com- modities.
When s=1, the corresponding aggregator or utility function is a Cobb–Douglas function.

When s approaches plus inﬁnity, the corresponding aggregator function f approaches a linear aggregator function which exhibits inﬁnite substitutability between each pair of inputs.
The CES unit cost function deﬁned by equa- tion (17.68) is not a fully ﬂexible functional form (unless the number of commodities n being aggregated is 2), but it is considerably more ﬂexible than the zero substitut- ability aggregator function (this is the special case of equation (17.68) where s is set equal to zero) that is exact for the Laspeyres and Paasche price indices.
17.63 Under the assumption of cost minimizing behaviour in period 0, Shephard’s Lemma (17.12), tells us that the observed ﬁrst period consumption of com- i , will be equal to u0 @c( p0)/@pi, where @c( p0)/ modity i, q0 @pi is the ﬁrst-order partial derivative of the unit cost function with respect to the ith commodity price eval- uated at the period 0 prices and u0=f (q0) is the aggre- gate (unobservable) level of period 0 utility.
Using the CES functional form deﬁned by equation (17.68) and assuming that s=1, the following equations are obtained: i =u0a0 q0 n P k=1 k)r ak( p0 (1=r)1 ai( p0 i )r1 for r 1s 6¼ 0 and i=1, 2, .

.

.

, n i )r1 u0c( p0)ai( p0 k)r ak( p0 = n P k=1 (17:69) 43 Thus homothetic preferences are assumed in this section.
44 In the mathematics literature, this aggregator function or utility function is known as a mean of order r; see Hardy, Littlewood and Po´ lya (1934, pp.

12–13).

327 CONSUMER PRICE INDEX MANUAL: THEORY AND PRACTICE These equations can be rewritten as: p0 i q0 i u0c( p0) = i )r ai( p0 k)r ak( p0 n P k=1 for i=1, 2, .

.

.
.

.

, n (17:70) where r:1s. Now consider the following Lloyd (1975) and Moulton (1996a) index number formula: PLM( p0, p1, q0, q1) P ( n i=1 )1=(1s) s0 i 1s p1 i p0 i for s 6¼ 1 (17:71) where s0 i, as usual: i is the period 0 expenditure share of commodity s0 i = i q0 p0 i n P kq0 p0 k k=1 p0 i q0 i u0c( p0) behaviour for i=1, 2, .

.

.
, n using the assumption of cost minimizing = i )r ai( p0 k)r ak( p0 n P k=1 using equation (17:70) (17:72) If equation (17.72) is substituted into equation (17.71), it is found that: PLM( p0, p1, q0, q1)= P n s0 i 1=r r p1 i p0 i i=1 8 >>< n = P >>: i=1 i )r ai( p0 k)r ak( p0 n P k=1 r p1 i p0 i 1=r 9 >>= >>; 1=r i )r ai( p1 k)r ak( p0 9 >>= >>; 8 >>< >>: = n P i=1 n P k=1 = a0 n P i=1 1=r i )r ai( p1 a0 n P k=1 1=r k)r ak( p0 = c( p1) c( p0) using r 1s and definition (17:68): (17:73) that shows 17.64 Equation (17.73) the Lloyd– Moulton index number formula PLM is exact for CES preferences.
Lloyd (1975) and Moulton (1996a) inde- pendently derived this result, but it was Moulton who appreciated the signiﬁcance of the formula (17.71) for statistical agency purposes.
Note that in order to eval- 328 uate formula (17.71) numerically, it is necessary to have information on: base period expenditure shares s0 i ; the price relatives p1 i between the base period and i =p0 the current period; and an estimate of the elasticity of substitution between the commodities in the aggregate, s. The ﬁrst two pieces of information are the standard information sets that statistical agencies use to evaluate the Laspeyres price index PL (note that PLM reduces to PL if s=0).
Hence, if the statistical agency is able to estimate the elasticity of substitution s based on past experience,45 then the Lloyd–Moulton price index can be evaluated using essentially the same information set that is used in order to evaluate the traditional Laspeyres index.
Moreover, the resulting CPI will be free of substitution bias to a reasonable degree of approximation.46 Of course, the practical problem with implementing this methodology is that estimates of the elasticity of substitution parameter s are bound to be somewhat uncertain, and hence the resulting Lloyd– Moulton index may be subject to charges that it is not objective or reproducible.

The statistical agency will have to balance the beneﬁts of reducing substitution bias with these possible costs.
Annual preferences and monthly prices 17.65 Recall the Lowe index, the deﬁnition of PLo( p0, p1, q), deﬁned by equation (15.15) in Chapter 15.

In paragraphs 15.33 to 15.64 of Chapter 15, it is noted that this formula is frequently used by statistical agen- cies as a target index for a CPI.

It is also noted that, while the price vectors p0 (the base period price vector) and p1 (the current period price vector) are monthly or quarterly price vectors, the quantity vector q:(q1, q2, .

.

.
.

.

, qn) which appears in this basket-type formula is usually taken to be an annual quantity vector that refers to a base year, b say, that is prior to the base period for the prices, month 0.
Thus, typically, a statistical agency will produce a CPI at a monthly frequency that has the form PLo( p0, pt, qb), where p0 is the price vector per- taining to the base period month for prices, month 0, pt 45 For the ﬁrst application of this methodology (in the context of the CPI), see Shapiro and Wilcox (1997a, pp.

121–123).
121–123).

They calculated superlative To¨ rnqvist indices for the United States for the years 1986– 95 and then calculated the Lloyd–Moulton CES index for the same period, using various values of s. They then chose the value of s (which was 0.7), which caused the CES index to most closely approximate the To¨ rnqvist index.

Essentially the same methodology was used by Alterman, Diewert and Feenstra (1999) in their study of United States import and export price indices.
For alternative methods for estimating s, see Balk (2000b).

46 What is a ‘‘reasonable’’ degree of approximation depends on the context.

Assuming that consumers have CES preferences is not a reasonable assumption in the context of estimating elasticities of demand: at least a second-order approximation to the consumer’s preferences is required in this context.
In the context of approximating changes in a consumer’s expenditures on the n commodities under it is usually adequate to assume a CES consideration, however, approximation.
THE ECONOMIC APPROACH TO INDEX NUMBER THEORY: THE SINGLE-HOUSEHOLD CASE is the price vector pertaining to the current period month for prices, month t say, and qb is a reference basket quantity vector that refers to the base year b, which is equal to or prior to month 0.47 The question to be addressed in the present section is: can this index be related to one based on the economic approach to index number theory?
The Lowe index as an approximation to a true cost of living index 17.66 Assume that the consumer has preferences deﬁned over consumption vectors q:[q1, .

.

.

, qn] that can be represented by the continuous increasing utility function f (q).

Thus if f (q1)>f (q0), then the consumer prefers the consumption vector q1 to q0.
Let qb be the annual consumption vector for the consumer in the base year b. Deﬁne the base year utility level ub as the utility level that corresponds to f (q) evaluated at qb: ub f (qb) (17:74) 17.67 For any vector of positive commodity prices p:[ p1, .

.

.
.

.

, pn] and for any feasible utility level u, the consumer’s cost function, C(u, p), can be deﬁned in the usual way as the minimum expenditure required to achieve the utility level u when facing the prices p: C(u, p) minq piqi : f (q1, .

.

.

, qn)=u : (17:75) n P i=1 1, .

.

.

, pb Let pb [ pb n] be the vector of annual prices that the consumer faced in the base year b.

Assume that the observed base year consumption vector qb [qb 1, .

.

.
.

.

, qb n] solves the following base year cost minimization pro- blem: C(ub, pb) minq n P i=1 i qi : f (q1, .

.

.

, qn)=ub pb i qb pb i n =P i=1 (17:76) The cost function will be used below in order to deﬁne the consumer’s cost of living price index.
17.68 Let p0 and pt be the monthly price vectors that the consumer faces in months 0 and t. Then the Konu¨ s true cost of living index, PK( p0, pt, qb), between months 0 and t, using the base year utility level ub=f (qb) as the reference standard of living, is deﬁned as the following ratio of minimum monthly costs of achieving the utility level ub: PK ( p0, pt, qb) C( f (qb), pt) C( f (qb), p0) (17:77) 17.69 Using the deﬁnition of the monthly cost minimization problem that corresponds to the cost C( f (qb), pt ), it can be seen that the following inequality holds: C( f (qb), pt) minq n P i=1 iqi : f (q1, .
.

.

, qn)=f (qb pt 1, .

.

.

, qb n) n P i=1 iqb pt i (17:78) since the base year quantity vector qb is feasible for the cost minimization problem.

Similarly, using the deﬁni- tion of the monthly cost minimization problem that corresponds to the month 0 cost C( f (qb), p0), it can be seen that the following inequality holds: C( f (qb), p0) minq n P i=1 n P i=1 i qb p0 i i qi : f (q1, .

.

.

, qn)=f (qb p0 1, .

.

.


, qb n) (17:79) since the base year quantity vector qb is feasible for the cost minimization problem.

17.70 It will prove useful to rewrite the two inequalities (17.78) and (17.79) as equalities.

This can be done if non-negative substitution bias terms, et and e0, are subtracted from the right-hand sides of these two inequalities.
Thus the inequalities (17.78) and (17.79) can be rewritten as follows: n C(ub, pt)= P i=1 iqb pt i et n C(ub, p0)= P i=1 i qb p0 i e0 (17:80) (17:81) 17.71 Using equations (17.80) and (17.81), and the deﬁnition (15.15) in Chapter 15 of the Lowe index, the following approximate equality for the Lowe index results: PLo( p0, pt, qb) iqb pt i n P i=1 n P i=1 C(ub, pt) C(ub, p0) i qb p0 i = fC(ub, pt)+etg fC(ub, p0)+e0g =PK ( p0, pt, qb) (17:82) Thus if the non-negative substitution bias terms e0 and et are small, then the Lowe index between months 0 and t, PLo( p0, pt, qb), will be an adequate approximation to the true cost of living index between months 0 and t, PK( p0, pt, qb).
17.72 A bit of algebraic manipulation shows that the Lowe index will be exactly equal to its cost of living counterpart if the substitution bias terms satisfy the following relationship:48 et e0 = C(ub, pt) C(ub, p0) =PK ( p0, pt, qb) (17:83) 47 As noted in Chapter 15, month 0 is called the price reference period and year b is called the weight reference period.

48 This assumes that e0 is greater than zero.


If e0 is equal to zero, then to have equality of PK and PLo, it must be the case that et is also equal to zero.
329 CONSUMER PRICE INDEX MANUAL: THEORY AND PRACTICE Equations (17.82) and (17.83) can be interpreted as fol- lows: if the rate of growth in the amount of substitution bias between months 0 and t is equal to the rate of growth in the minimum cost of achieving the base year utility level ub between months 0 and t, then the obser- vable Lowe index, PLo( p0, pt, qb), will be exactly equal to its true cost of living index counterpart, PK( p0, pt, qb).49 to know whether condition (17.83) will hold or whether the substitution bias terms e0 and et will be small.
Thus, ﬁrst-order and second- order Taylor series approximations to these substitution bias terms are developed in paragraphs 17.74 to 17.83. is diﬃcult 17.73 It A ﬁrst-order approximation to the bias of the Lowe index 17.74 The true cost of living index between months 0 and t, using the base year utility level ub as the reference is the ratio of two unobservable costs, utility level, C(ub, pt)/C(ub, p0).
However, both of these hypothetical costs can be approximated by ﬁrst-order Taylor series approximations that can be evaluated using observable information on prices and base year quantities.
The ﬁrst-order Taylor series approximation to C(ub, pt) around the annual base year price vector pb is given by the following approximate equation:50 [ pt ipb i ] @C(ub, pb) @pi n C(ub, pt) C(ub, pb)+ P i=1 n =C(ub, pb)+P i=1 i [ pt qb ipb i ] using assumption (17:76) and Shephard’s Lemma (17.12) n =P i=1 n =P i=1 n i qb pb i +P i=1 i [ pt qb ipb i ] using (17:76) iqb pt i : (17:84) Similarly, the ﬁrst-order Taylor series approximation to C(ub, p0) around the annual base year price vector pb is given by the following approximate equation: n C(ub, p0) C(ub, pb)+ P i=1 n =C(ub, pb)+ P i=1 n @C(ub, pb) @pi [ p0 i pb i ] qb i ½p0 i pb i i qb pb i +P qb i ½p0 i pb i i=1 n =P i=1 n =P i=1 i qb p0 i (17:85) 49 It can be seen that, when month t is set equal to month 0, et=e0 and C(ub, pt)=C(ub, p0), and thus equation (17.83) is satisﬁed and PLo=PK.
This is not surprising since both indices are equal to unity when t=0.

50 This type of Taylor series approximation was used in Schultze and Mackie (2002, p. 91) in the cost of living index context, but it essen- tially dates back to Hicks (1941–42, p. 134) in the consumer surplus context.

See also Diewert (1992b, p. 568) and Hausman (2002, p. 8).
330 17.75 Comparing approximate equation (17.84) with equation (17.80), and comparing approximate equation (17.85) with equation (17.81), it can be seen that, to the accuracy of the ﬁrst-order approximations used in (17.84) and (17.85), the substitution bias terms et and e0 will be zero.
Using these results to reinterpret the approximate equation (17.82), it can be seen that if the month 0 and month t price vectors, p0 and pt, are not too diﬀerent from the base year vector of prices pb, then the Lowe index PLo( p0, pt, qb) will approximate the true cost of living index PK( p0, pt, qb) to the accuracy of a ﬁrst- order approximation.
This result is quite useful, since it indicates that if the monthly price vectors p0 and pt are just randomly ﬂuctuating around the base year prices pb (with modest variances), then the Lowe index will serve as an adequate approximation to a theoretical cost of living index.
However, if there are systematic long-term trends in prices and month t is fairly distant from month 0 (or the end of year b is quite distant from month 0), then the ﬁrst-order approximations given by approx- imate equations (17.84) and (17.85) may no longer be adequate and the Lowe index may have a considerable bias relative to its cost of living counterpart.

The hypothesis of long-run trends in prices will be explored in paragraphs 17.76 to 17.83.
A second-order approximation to the substitution bias of the Lowe index 17.76 A second-order Taylor series approximation to C(ub, pt) around the base year price vector pb is given by the following approximate equation: n C(ub, pt) C(ub, pb)+ P i=1 [ pt ipb i ] [ pt ipb i ][ pt jpb j ] @C(ub, pb) @pi @2C(ub, pb) @pi@pj n P j=1 @2C(ub, pb) @pi@pj (17:86) + 1 2 n P i=1 i qb pb i + n =P i=1 [ pt ipb n P j=1 n 1 P 2 i=1 jpb i ][ pt j ] where the last equality follows using approximate equa- tion (17.84).51 Similarly, a second-order Taylor series approximation to C(ub, p0) around the base year price vector pb is given by the following approximate equation: n C(ub, p0) C(ub, pb)+ P i=1 + 1 2 n P i=1 i qb p0 i + n =P i=1 [ p0 i pb n P j=1 1 2 i ][ p0 n P i=1 j pb j ] i pb i ] [ p0 @C(ub, pb) @pi @2C(ub, pb) @pi@pj @2C(ub, pb) @pi@pj [ p0 i pb n P j=1 i ][ p0 j pb j ] (17:87) 51 This type of second-order approximation is attributable to Hicks (1941–42, pp.
133–134; 1946, p. 331).

See also Diewert (1992b, p. 568), Hausman (2002, p. 18) and Schultze and Mackie (2002, p. 91).

For alternative approaches to modelling substitution bias, see Diewert (1998a; 2002c, pp.

598–603) and Hausman (2002).

THE ECONOMIC APPROACH TO INDEX NUMBER THEORY: THE SINGLE-HOUSEHOLD CASE where the last equality follows using the approximate equation (17.85).
17.77 Comparing approximate equation (17.86) with equation (17.80), and approximate equation (17.87) with equation (17.81), it can be seen that, to the the accuracy of a second-order approximation, month 0 and month t substitution bias terms, e0 and et, will be equal to the following expressions involving the second-order partial derivatives of the consumer’s the cost base year standard of living ub and at the base year prices pb: function @2C(ub, pb)/@pi@pj evaluated at e0 1 2 n P i=1 n P j=1 @2C(ub, pb) @pi@pj [ p0 i pb i ][ p0 j pb j ] et 1 2 n P i=1 n P j=1 @2C(ub, pb) @pi@pj [ pt ipb i ][ pt jpb j ] (17:88) (17:89) Since the consumer’s cost function C(u, p) is a concave function in the components of the price vector p,52 it is known53 that the n by n (symmetric) matrix of second- order partial derivatives [@2C(ub, pb)/@pi@pj] is negative semi-deﬁnite.54 Hence, for arbitrary price vectors pb, p0 and pt, the right-hand sides of approximations (17.88) and (17.89) will be non-negative.
Thus, to the accuracy of a second-order approximation, the substitution bias terms e0 and et will be non-negative.

17.78 Now assume that there are long-run sys- tematic trends in prices.

Assume that the last month of the base year for quantities occurs M months prior to month 0, the base month for prices, and assume that prices trend linearly with time, starting with the last month of the base year for quantities.

Thus, assume the existence of constants aj for j=1, .

.

.
.

.

, n such that the price of commodity j in month t is given by: i=pb pt i +aj(M+t) for j=1, .

.

.

, n and t=0, 1, .

.

.
, T (17:90) into approximations Substituting equation (17.90) (17.88) and (17.89) leads to the following second-order approximations to the two substitution bias terms, e0 and et:55 where g is deﬁned as follows: g 1 2 n P i=1 n P j=1 @2C(ub, pb) @pi@pj aiaj 0 (17:93) 17.79 It should be noted that the parameter g will be zero under two sets of conditions:56 All the second-order partial derivatives of the con- sumer’s cost function @2C(ub, pb)/@pi @pj are equal to zero.
Each commodity price change parameter aj is pro- portional to the corresponding commodity j base year price pb j .57 The ﬁrst condition is empirically unlikely since it implies that the consumer will not substitute away from com- modities of which the relative price has increased.

The second condition is also empirically unlikely, since it implies that the structure of relative prices remains unchanged over time.

Thus, in what follows, it will be assumed that g is a positive number.
17.80 In order to simplify the notation in what fol- lows, deﬁne the denominator and numerator of the month t Lowe index, PLo(p0, pt, qb), as a and b respec- tively; i.e., deﬁne: n a P i=1 i qb p0 i (17:94) iqb pt i n b P i=1 Using equation (17.90) to eliminate the month 0 prices i from equation (17.94) and the month t prices pt p0 i from equation (17.95) leads to the following expressions for a and b: (17:95) n a=P i=1 n i qb pb i +P i=1 aiqb i M (17:96) n b=P i=1 n i qb pb i +P i=1 aiqb i (M+t) (17:97) It is assumed that a and b58 are positive and that n P i=1 aiqb i 0 (17:98) e0 gM2 et g(M+t)2 (17:91) (17:92) Assumption (17.98) rules out a general deﬂation in prices.
17.81 Deﬁne the bias in the month t Lowe index, Bt, as the diﬀerence between the true cost of living index PK( p0, pt, qb) deﬁned by equation (17.77) and the i=1 Pn 52 See Diewert (1993b, pp.

109–110).

53 See Diewert (1993b, p. 149).

54 A symmetric n by n matrix A with ijth element equal to aij is negative semi-deﬁnite if, and only if for every vector z:[z1, .

.

.
.

.

, zn], it is the case that Pn 55 Note that the period 0 approximate bias deﬁned by the right-hand side of approximation (17.91) is ﬁxed, while the period t approximate bias deﬁned by the right-hand side of (17.92) increases quadratically with time t. Hence, the period t approximate bias term will eventually overwhelm the period 0 approximate bias in this linear time trends case, if t is allowed to become large enough.

j=1aijzizj 0.
j=1aijzizj 0.

56 A more general condition that ensures the positivity of g is that the vector [a1, .

.

.

, an] is not an eigenvector of the matrix of second-order partial derivatives @ 2C(u, p)/@pi@pj that corresponds to a zero eigen- value.

57 It is known that C(u, p) is linearly homogeneous in the components of the price vector p; see Diewert (1993b, p. 109) for example.
Hence, using Euler’s Theorem on homogeneous functions, it can be shown that pb is an eigenvector of the matrix of second-order partial deriva- tives @2C(u, p)/@pi@pj that corresponds to a zero eigenvalue and thus Pn j =0; see Diewert (1993b, p. 149) for a detailed proof of this result.

58 It is also assumed that agM2 is positive.
j=1[@2C(u, p)=@pi@pj]pb i pb Pn i=1 331 CONSUMER PRICE INDEX MANUAL: THEORY AND PRACTICE corresponding Lowe index PLo( p0, pt, qb): these assumptions means that: Bt PK ( p0, pt, qb)PLo( p0, pt, qb) = C(ub, pt) C(ub, p0) b a using equations (17:94) and (17:95) b a [bet] [ae0] = using equations (17:80) and (17:81) [bg(M+t)2] agM2 b a =g using equations (17:91) and (17:92) f(ba)M22aMtat2g fa[agM2]g aiqb i t n P i=1 n n M22 P i=1 fa[agM2]g i qb pb i + P i=1 =g simplifying terms aiqb i M Mtat2 using equations (17:96) and (17:97) n P i=1 aiqb i t n M22 P i=1 i qb pb i fa[agM2]g =g using equation (17:98): Mt+at2 < 0 (17:99) Thus, for t 1, the Lowe index will have an upward bias (to the accuracy of a second-order Taylor series approximation) compared to the corresponding true cost of living index PK( p0, pt, qb), since the approximate bias deﬁned by the last expression in equation (17.99) is the sum of one non-positive and two negative terms.
Moreover, this approximate bias will grow quadratically in time t.59 17.82 In order to give the reader some idea of the magnitude of the approximate bias Bt deﬁned by the last line of equation (17.99), a simple special case will be considered at this point.

Suppose there are only two commodities and that, at the base year, all prices and quantities are equal to 1.

Thus, pb i =1 for i=1, 2 and Pn i =2.


Assume that M=24 so that the base year data on quantities take two years to process before the Lowe index can be implemented.

Assume that the monthly rate of growth in price for commodity 1 is a1=0.002 so that after one year, the price of commodity 1 rises 0.024 or 2.4 per cent.

Assume that commodity 2 falls in price each month with a2=0.002 so that the price of commodity 2 falls 2.4 per cent in the ﬁrst year after the base year for quantities.
Thus the relative price of the two commodities is steadily diverging by about 5 per cent per year.

Finally, assume that @2C(ub, pb)/ @p1@p1=@2C(ub, pb)/@p2@p2=1 and @2C(ub, pb)/@p1 @p2=@2C(ub, pb)/@p2@p1=1.

These assumptions imply that the own elasticity of demand for each commodity is 1 at the base year consumer equilibrium.
Making all of i=1 pb i =qb i qb 59 If M is large relative to t, then it can be seen that the ﬁrst two terms in the last equation of (17.99) can dominate the last term, which is the quadratic in t term.
332 n 2=P i=1 i qb pb i =a=b P aiqb i =0 M=24; g=0:000008 n i=1 (17:100) Substituting the parameter values deﬁned in equation (17.100) into equation (17.99) leads to the following formula for the approximate amount that the Lowe index will exceed the corresponding true cost of living index at month t: Bt=0:000008 (96t+2t2) 2(20:004608) (17:101) Evaluating equation (17.101) at t=12, t=24, t=36, t=48 and t=60 leads to the following estimates for Bt: 0.0029 (the approximate bias in the Lowe index at the end of the ﬁrst year of operation for the index); 0.0069 (the bias after two years); 0.0121 (the bias after three years); 0.0185 (the bias after four years); 0.0260 (the bias after ﬁve years).
Thus, at the end of the ﬁrst year of the operation, the Lowe index will only be above the cor- responding true cost of living index by approximately a third of a percentage point but, by the end of the ﬁfth year of operation, it will exceed the corresponding cost of living index by about 2.6 percentage points, which is no longer a negligible amount.60 17.83 The numerical results in the previous para- graph are only indicative of the approximate magnitude of the diﬀerence between a cost of living index and the corresponding Lowe index.
The important point to note is that, to the accuracy of a second-order approximation, the Lowe index will generally exceed its cost of living counterpart.

The results also indicate, however, that this diﬀerence can be reduced to a negligible amount if: the lag in obtaining the base year quantity weights is minimized; and the base year is changed as frequently as possible.
It should also be noted that the numerical results depend on the assumption that long-run trends in prices exist, which may not be true,61 and on elasticity assumptions that may not be justiﬁed.62 Statistical agencies should prepare their own carefully constructed estimates of the diﬀerences between a Lowe index and a cost of living index in the light of their own particular circumstances.
The problem of seasonal commodities 17.84 The assumption that the consumer has annual preferences over commodities purchased in the base year 60 Note that the relatively large magnitude of M compared to t leads to a bias that grows approximately linearly with t rather than quad- ratically.

61 For mathematical convenience, the trends in prices were assumed to be linear, rather than the more natural assumption of geometric trends in prices.
62 Another key assumption that was used to derive the numerical results is the magnitude of the divergent trends in prices.

If the price divergence vector is doubled to a1=0.004 and a2=0.004, then the parameter g quadruples and the approximate bias will also quadruple.
THE ECONOMIC APPROACH TO INDEX NUMBER THEORY: THE SINGLE-HOUSEHOLD CASE for the quantity weights, and that these annual pref- erences can be used in the context of making monthly purchases of the same commodities, was a key one in relating the economic approach to index number theory to the Lowe index.

This assumption that annual pref- erences can be used in a monthly context is, however, somewhat questionable because of the seasonal nature of some commodity purchases.
The problem is that it is very likely that consumers’ preference functions sys- tematically change as the season of the year changes.

National customs and weather changes cause house- holds to purchase certain goods and services during some months and not at all for other months.

For example, Christmas trees are purchased only in jackets are not usually purchased December and ski during summer months.
Thus, the assumption that annual preferences are applicable during each month of the year is only acceptable as a very rough approxima- tion to economic reality.

17.85 The economic approach to index number theory can be adapted to deal with seasonal preferences.
The simplest economic approach is to assume that the consumer has annual preferences over commodities classiﬁed not only by their characteristics but also by the month of purchase.63 Thus, instead of assuming that the consumer’s annual utility function is f (q) where q is an n-dimensional vector, assume that the consumer’s annual utility function is F [ f 1(q1), f 2(q2), .

.

.
.

.

, f 12(q12)] where q1 is an n-dimensional vector of commodity pur- chases made in January, q2 is an n-dimensional vector of commodity purchases made in February, .

.

.

, and q12 is an n-dimensional vector of commodity purchases made in December.64 The sub-utility functions f 1, f 2, .

.

.

, f 12 represent the consumer’s preferences when making purchases in January, February, .

.

.

., and December, respectively.


These monthly sub-utilities can then be aggregated using the macro-utility function F in order to deﬁne overall annual utility.
It can be seen that these assumptions on preferences can be used to justify two types of cost of living index: an annual cost of living index that compares the prices in all months of a current year with the corresponding monthly prices in a base year;65 and 12 monthly cost of living indices where the index for month m compares the prices of month m in the current year with the prices of month m in the base year for m=1, 2, .

.

.
.

.

, 12.66 17.86 The annual Mudgett–Stone indices compare costs in a current calendar year with the corresponding costs in a base year.

However, any month could be chosen as the year-ending month of the current year, and 63 This assumption and the resulting annual indices were ﬁrst proposed by Mudgett (1955, p. 97) and Stone (1956, pp.

74–75).

64 If some commodities are not available in certain months m, then those commodities can be dropped from the corresponding monthly quantity vectors qm.
65 For further details on how to implement this framework, see Mudgett (1955, p. 97), Stone (1956, pp.

74–75) and Diewert (1998b, pp.

459 – 460).

66 For further details on how to implement this framework, see Diewert (1999a, pp.

50–51).
50–51).

the prices and quantities of this new non-calendar year could be compared to the prices and quantities of the base year, where the January prices of the non-calendar year are matched to the January prices of the base year, the February prices of the non-calendar year are matched to the February prices of the base year, and so on.
If further assumptions are made on the macro-utility function F, then this framework can be used in order to justify a third type of cost of living index: a moving year annual index.67 This index compares the cost over the past 12 months of achieving the annual utility achieved in the base year with the base year cost, where the January costs in the current moving year are matched to January costs in the base year, the February costs in the current moving year are matched to February costs in the base year, and so on.
These moving year indices can be calculated for each month of the current year and the resulting series can be interpreted as (uncentred) seasonally adjusted (annual) price indices.68 17.87 It should be noted that none of the three types of indices described in the previous two paragraphs is suitable for describing the movements of prices going from one month to the following month; i.e., they are not suitable for describing short-run movements in inﬂation.

This is obvious for the ﬁrst two types of index.


To see the problem with the moving year indices, con- sider a special case where the bundle of commodities purchased in each month is entirely speciﬁc to each month.
Then it is obvious that, even though all the above three types of index are well deﬁned, none of them can describe anything useful about month-to-month changes in prices, since it is impossible to compare like with like, going from one month to the next, under the hypotheses of this special case.

It is impossible to com- pare the incomparable.

17.88 Fortunately, it is not the case that household purchases in each month are entirely speciﬁc to the month of purchase.
Thus month-to-month price com- parisons can be made if the commodity space is restricted to commodities that are purchased in each month of the year.
This observation leads to a fourth type of cost of living index, a month-to-month index, deﬁned over commodities that are available in every month of the year.69 This model can be used to justify the economic approach described in paragraphs 17.66 to 17.83. Commodities that are purchased only in certain months of the year, however, must be dropped from the scope of the index.
Unfortunately, it is likely that consumers have varying monthly preferences over the commodities that are always available and, if this is the case, the month-to-month cost of living index (and the corresponding Lowe index) deﬁned over always- available commodities will generally be subject to sea- sonal ﬂuctuations.

This will limit the usefulness of the 67 See Diewert (1999a, pp.

56–61) for the details of this economic approach.

68 See Diewert (1999a, pp.


67–68) for an empirical example of this approach applied to quantity indices.

An empirical example of this moving year approach to price indices is presented in Chapter 22.

69 See Diewert (1999a, pp.

51–56) for the assumptions on preferences that are required in order to justify this economic approach.
333 CONSUMER PRICE INDEX MANUAL: THEORY AND PRACTICE index as a short-run indicator of general inﬂation since it will be diﬃcult to distinguish a seasonal movement in the index from a systematic general movement in pri- ces.70 Note also that if the scope of the index is restricted to always-available commodities, then the resulting month-to-month index will not be comprehensive, whereas the moving year indices will be comprehensive in the sense of using all the available price information.
17.89 The above considerations lead to the conclu- sion that it may be useful for statistical agencies to produce at least two consumer price indices: a moving year index which is comprehensive and seasonally adjusted, but which is not necessarily useful for indicating month-to-month changes in general inﬂation; and a month-to-month index which is restricted to non- seasonal commodities (and hence is not comprehen- sive), but which is useful for indicating short-run movements in general inﬂation.
The problem of a zero price increasing to a positive price 17.90 In a recent paper, Haschka (2003) raised the problem of what to do when a price which was pre- viously zero is increased to a positive level.

He gave two examples for Austria, where parking and hospital fees were raised from zero to a positive level.
In this situa- tion, it turns out that basket-type indices have an advantage over indices that are weighted geometric averages of price relatives, since basket-type indices are well deﬁned even if some prices are zero.

i and quantities qt 17.91 The problem can be considered in the context of evaluating the Laspeyres and Paasche indices.
Suppose as usual that the prices pt i of the ﬁrst n commodities are positive for periods 0 and 1, but that the price of commodity n+1 in period 0 is zero but is positive in period 1.

In both periods, the consumption of com- modity n+1 is positive.
Thus the assumptions on the prices and quantities of commodity n+1 in the two per- iods under consideration can be summarized as follows: n+1 > 0 n+1=0 p1 p0 q0 n+1 > 0 Typically, the increase in price of commodity n+1 from its initial non-zero level will cause consumption to fall so n+1 < q0 that q1 n+1, but this inequality is not required for the analysis below.
q1 n+1 > 0 (17:102) 17.92 Let the Laspeyres index between periods 0 and 1, restricted to the ﬁrst n commodities, be denoted as Pn L and let the Laspeyres index, deﬁned over all n+1 com- modities, be deﬁned as Pn+1 i denote .

Also let v0 i p0 i q0 L 70 One problem with using annual weights in the context of seasonal movements in prices and quantities is that a change in price when a commodity is out of season can be greatly magniﬁed by the use of annual weights.
Baldwin (1990, p. 251) noted this problem with an annual weights price index: ‘‘But a price index is adversely aﬀected if any seasonal good has the same basket share for all months of the year; the good will have an inappropriately small basket share in its in season months, an inappropriately large share in its oﬀ season months.’’ Seasonality problems are considered again from a more pragmatic point of view in Chapter 22.

334 the value of expenditures on commodity i in period 0.
Then by the deﬁnition of the Laspeyres index deﬁned over all n+1 commodities: Pn+1 L n+1 P i=1 n+1 P i=1 i q0 p1 i i q0 p0 i =Pn L+ n+1 n+1q0 p1 n P v0 i i=1 (17:103) where p0 n+1=0 was used in order to derive the second equation above.
Thus the complete Laspeyres index Pn+1 deﬁned over all n+1 commodities is equal to the L incomplete Laspeyres index Pn L (which can be written in traditional price relative and base period expenditure share form), plus the mixed or hybrid expenditure n+1q0 p1 n+1 divided by the base period expenditure on the ﬁrst n commodities, Pn i=1v0 i .
Thus the complete Laspeyres index can be calculated using the usual information available to the price statistician plus two additional pieces of information: the new non-zero price for commodity n+1 in period 1, p1 n+1, and an estimate of consumption of commodity n+1 in period 0 (when it was free), q0 n+1.
Since it is often governments that change the previously zero price to a positive price, the decision to do this is usually announced in advance, which will give the price statistician an opportunity to form an estimate for the base period demand, q0 17.93 Let the Paasche index between periods 0 and 1, restricted to the ﬁrst n commodities, be denoted as Pn P and let the Paasche index, deﬁned over all n+1 com- modities, be deﬁned as Pn+1 i denote the value of expenditures on commodity i in period 1.
Then, by the deﬁnition of the Paasche index deﬁned over all n+1 commodities: .

Also let v1 i p1 i q1 P n+1.

Pn+1 P n+1 P i=1 n+1 P i=1 i q1 p1 i p0 i q1 i =P n P+ v1 n+1 n P i=1 p0 i q1 i =P n P+ v1 n+1 i =( p1 v1 i =p0 i ) n P i=1 (17:104) where p0 n+1=0 was used in order to derive the second equation above.
Thus the complete Paasche index Pn+1 P deﬁned over all n+1 commodities is equal to the incomplete Paasche index Pn P (which can be written in traditional price relative and current period expenditure share form), plus the current period expenditure on commodity n+1, v1 n+1, divided by a sum of current period expenditures on the ﬁrst n commodities, v1 i , divided i =p0 by the ith price relative for the ﬁrst n commodities, p1 i .
Thus the complete Paasche index can be calculated using THE ECONOMIC APPROACH TO INDEX NUMBER THEORY: THE SINGLE-HOUSEHOLD CASE the usual information available to the price statistician plus information on current period expenditures.
17.94 Once the complete Laspeyres and Paasche indices have been calculated using equations (17.103) and (17.104), then the complete Fisher index can be calculated as the square root of the product of these two indices: P n+1 F =[P n+1 L P n+1 P ]1=2 (17:105) It should be noted that the complete Fisher index deﬁned by equation (17.105) satisﬁes the same exact index number results as were demonstrated in para- graphs 17.27 to 17.32 above; i.e., the Fisher index remains a superlative index even if prices are zero in one period but positive in the other.
Thus the Fisher price index remains a suitable target index even in the face of zero prices.

335 THE ECONOMIC APPROACH TO INDEX NUMBER THEORY: THE MANY-HOUSEHOLD CASE 18 Introduction 18.1 In the previous chapter on the economic approach to index numbers, it was implicitly assumed that the economy behaved as if there were a single representative consumer.

In the present chapter, the eco- nomic approach is extended to an economy with many household groups or many regions.
In the algebra below, an arbitrary number of households, H say, is considered.

In principle, each household in the economy under consideration could have its own consumer price index.

In practice, however, it will be necessary to group households into various classes.

Within each class, it will be necessary to assume that the group of households in the class behaves as if it were a single household in order to apply the economic approach to index number theory.
The partition of the economy into H household classes can also be given a regional interpretation: each house- hold class could be interpreted as a group of households within a region of the country under consideration.

18.2 The concepts of a plutocratic index and a con- ditional index are introduced in paragraphs 18.3 to 18.13.
Using the plutocratic concept, each household in the economy is given a weight in the national index that is proportional to the household’s expenditures on com- modities for the two periods under consideration.

A conditional index is an index that depends on environ- mental variables that might aﬀect household expendi- tures on commodities.

One example of an environmental variable is the weather: if the weather is cold, then households will spend more on heating fuel.
In para- graphs 18.14 to 18.22 it is shown how a national Fisher price index can approximate a plutocratic cost of living index.

Finally, paragraphs 18.23 to 18.35 consider an alternative conceptual framework for a national index, the democratic index.

Using this index concept, each household in the economy is given an equal weight in the national index (as opposed to the plutocratic concept where households that spend more get a higher weight in the national index).
Plutocratic cost of living indices and observable bounds 18.3 In this section, an economic approach to the consumer price index (CPI) is considered that is based on the plutocratic cost of living index that was originally deﬁned by Prais (1959).
This concept was further reﬁned by Pollak (1980, p. 276; 1981, p. 328), who deﬁned his Scitovsky–Laspeyres cost of living index as the ratio of total expenditure required to enable each household in the economy under consideration to attain its base period indiﬀerence surface at period 1 prices to the corresponding expenditure required to attain the same standard of living using period 0 prices.

In the following paragraph, this concept will be explained more fully.

h1, pt h ( pt h2, .

.

.


, pt 18.4 Suppose that there are H households (or regions) in the economy and suppose further that there are n commodities in the economy in periods 0 and 1 that households consume and that we wish to include in our deﬁnition of the cost of living.

Denote an n- dimensional vector of commodity consumption in a given period by q:(q1, q2, .

.

.

, qn) as usual.

Denote the vector of period t market prices faced by household h by pt hN) for t=0,1.
Note that it is not assumed that each household faces the same vector of commodity prices.

In addition to the market com- modities that are in the vector q, it is assumed that each household is aﬀected by an M-dimensional vector of environmental1 or demographic2 variables or public goods, e:(e1, e2, .

.

.

, eM).
.

.

, eM).

It is supposed that there are H households (or regions) in the economy during periods 0 and 1, and the preferences of household h over diﬀerent combinations of market commodities q and environmental variables e can be represented by h= the 1, 2, .

.

.

, H.3 For periods t=0,1 and for households h=1, 2, .

.

.
.

.

, H, it is assumed that the observed house- hold h consumption vector qt is a solution to the following household h expenditure minimization problem: function f h(q, e) continuous utility h1, .

.

.

, qt h (qt hN) for minq fpt h) ut hq: f h(q, et t=0, 1; h=1, 2, .

.

.
.

.

, H hg Ch(ut h, et h, pt h); (18:1) where et h is the environmental vector facing household h in period t, ut h) is the utility level achieved by household h during period t and Ch is the cost or expenditure function that to the utility function f h.4 Basically, these assumptions mean that h f h(qt is dual h, et 1 This is the terminology used by Pollak (1989, p. 181) in his model of the conditional cost of living concept.
2 Caves, Christensen and Diewert (1982a, p. 1409) used the terms demographic variables or public goods to describe the vector of con- ditioning variables e in their generalized model of the Konu¨ s price index or cost of living index, while Diewert (2001) used the term environmental variables.

3 It is assumed that each f h(q, e) is continuous and increasing in the components of q and e, and is concave in the components of q.

4 In order pq=Pn rather than the usual summation notation.
section the notation i=1piqi as the inner product between the vectors p and q is used to simplify notation, in this 337 CONSUMER PRICE INDEX MANUAL: THEORY AND PRACTICE each household has stable preferences over the same list of commodities during the two periods under con- sideration, the same households appear in each period and each household chooses its consumption bundle in the most cost-eﬃcient way during each period, condi- tional on the environmental vector that it faces during each period.
Note again that the household (or regional) prices are in general diﬀerent across households (or regions).

18.5 With the above assumptions in mind, the example of Pollak (1980; 1981) and Diewert (1983a, p. 190)5 is followed.

The class of conditional pluto- living indices, P*( p0, p1, u, e1, e2, .

.

.

, cratic cost of eH), pertaining to periods 0 and 1 for the arbitrary utility vector of household utilities u:(u1, u2, .

.

.
.

.

, uH) and for the arbitrary vectors of household environ- mental variables eh for h=1, 2, .

.

.

, H is deﬁned as follows: P*( p0 H, p1 1, .

.

.

, p1 H, u, e1, e2, .

.

.

, eH) 1, .

.

.
, p0 H P h=1 H P h=1 Ch(uh, eh, p1 h) Ch(uh, eh, p0 h) (18:2) The numerator on the right-hand side of equation (18.2) is the sum over households of the minimum cost, Ch(uh, eh, p1 h), for household h to achieve the arbitrary utility level uh, given that the household h faces the arbitrary vector of household h environmental variables eh and also faces the period 1 vector of prices p1 h. The denominator on the right-hand side of equation (18.2) is the sum over households of the minimum cost, Ch(uh, eh, p0 h), for household h to achieve the same arbi- trary utility level uh, given that the household faces the same arbitrary vector of household h environmental variables eh and also faces the period 0 vector of prices p0 h. Thus in the numerator and denominator of equation (18.2), only the price variables are diﬀerent, which is precisely what is wanted in a theoretical deﬁnition of a consumer price index.
1, u0 18.6 The general deﬁnition (18.2) is now specialized by replacing the general utility vector u by either the period 0 vector of household utilities u0 (u0 2, .

.

.

u0 H) or the period 1 vector of household utilities u1 (u1 1, u1 2, .

.

.

u1 H).

The general deﬁnition is also specialized by replacing the general household environmental vectors (e1, e2, .

.

.

eH) : e by either the period 0 vector of 2, .

.

.
.

.

e0 household environmental variables e0 (e0 H) or the period 1 vector of household environmental variables e1 (e1 H).

The choice of the base period vector of utility levels and base period environ- mental variables leads to the Laspeyres conditional plu- 2, .

.

.

, e1 1, e0 1, e1 5 These authors provided generalizations of the plutocratic cost of living index attributable to Prais (1959).
Pollak and Diewert did not include the environmental variables in their deﬁnitions of a group cost of living index.

338 1, .

.

.

, p0 living index, P*( p0 tocratic cost of 1, .

.

.

, H, u0, e0).6 The choice of the period 1 vector of utility p1 levels and period 1 environmental variables leads to the Paasche conditional plutocratic cost of living index, P*( p0 H, u1, e1).

It turns out that these 1, .

.

.

, p1 last two indices satisfy some interesting inequalities, which are derived below.

1, .

.
1, .

.

.

, p0 H, p1 H, p1 18.7 Using deﬁnition (18.2), the Laspeyres condi- 1, .

.

.

, p0 H, tional plutocratic cost of living index, P*( p0 1, .

.

.

, p1 p1 H, u0, e0), may be written as follows: P*( p0 H, p1 1, .

.

.

, p1 H, u0, e0 1, e0 2, .

.

.

, e0 H) 1, .

.

.


, p0 H P h=1 H P h=1 Ch(u0 h, e0 h, p1 h) Ch(u0 h, e0 h, p0 h) H P h=1 = Ch(u0 h, e0 h, p1 h) using equation (18:1) for t=0 H P h=1 hq0 p0 h H P h=1 H P h=1 hq0 p1 h hq0 p0 h (18:3) h, e0 h, p1 since Ch(u0 h and q0 h is feasible for the cost minimization problem for h=1, 2, .

.

.
.

.

, H h) minq fp1 hq: f h(q, e0 hg p1q0 h) u0 PPL where PPL is deﬁned to be the observable (in principle) plutocratic Laspeyres price index, PH hq0 h, which uses the individual vectors of household or regional quantities for period 0, (q0 H), as quantity weights.7 18.8 If prices are equal across households 1, .

.

.

, q0 h= PH h=1 p0 h=1p1 hq0 (or regions), so that h=pt pt for t=0, 1 and h=1, 2, .

.

.
.

.

, H, (18:4) then the plutocratic (or disaggregated) Laspeyres price index, PPL, collapses down to the usual aggregate Las- peyres index, PL; i.e., then PPL becomes 6 This is the concept of a cost of living index that Triplett (2001) found most useful for measuring inﬂation: ‘‘One might want to produce a COL conditional on the base period’s weather experience.

.

.

.

In this case, the unusually cold winter does not aﬀect the conditional COL subindex that holds the environment constant.

.

.

.
.

.

.

the COL subindex that holds the environment constant is probably the COL concept that is most useful for an anti-inﬂation policy.’’ Hill (1999, p. 4) endorsed this point of view.

7 Thus the plutocratic Laspeyres index can be regarded as an ordinary Laspeyres index except that each commodity consumed by each household (or in each region) is regarded as a separate commodity.
THE ECONOMIC APPROACH TO INDEX NUMBER THEORY: THE MANY-HOUSEHOLD CASE PPL = H P h=1 H P h=1 hq0 p1 h hq0 p0 h H p1 P h=1 H p0 P h=1 q0 h q0 h = p1q0 p0q0 PL (18:5) where the total quantity vector in period t is deﬁned as H qt P h=1 qt h for t=0, 1 (18:6) 18.11 If prices are equal across households (or re- gions), so that assumptions (18.4) hold, then the dis- aggregated Paasche price index PPP collapses down to the usual aggregate Paasche index, PP; i.e., then PPP becomes PPP = H P h=1 H P h=1 hq1 p1 h p0 hq1 h H p1 P h=1 H p0 P h=1 q1 h q1 h = p1q1 p0q1 PP (18:8) H, p1 1, .
.

.

, p1 1, .

.

.

, p0 18.9 The inequality (18.3) says that the theoretical Laspeyres plutocratic conditional cost of living index, P*( p0 H, u0, e0), is bounded from above by the observable (in principle) plutocratic or disag- gregated Laspeyres price index, PPL.
The special case of inequality (18.3) when the equal prices assumption (18.4) holds8 was ﬁrst obtained by Pollak (1989, p. 182) for the case of one household with environmental variables and by Pollak (1980, p. 276) for the many-household case, but where the environmental variables are absent from the household utility and cost functions.

18.10 In a similar manner, specializing deﬁnition (18.2), the Paasche conditional plutocratic cost of living H, u1, e1), may be written 1, .

.

.
.

.

, p1 index, P*( p0 as follows: 1, .

.

.

, p0 H, p1 P*( p0 H, p1 1, .

.

.

, p1 H, u1, e1 1, e1 2, .

.

.

, e1 H) 1, .

.

.


, p0 H P h=1 H P h=1 Ch(u1 h, e1 h, p1 h) Ch(u1 h, e1 h, p0 h) H P h=1 hq1 p1 h using equation (18:1) for t=1 Ch(u1 h, e1 h, p0 h) = H P h=1 hq1 p1 h p0 hq1 h H P h=1 H P h=1 PPP using a feasibility argument (18:7) where PPP is deﬁned to be the plutocratic or dis- aggregated (over households) Paasche price index, PH h, which uses the individual vectors of household quantities for period 1, (q1 H), as quantity weights.

1, .

.

.
1, .

.

.

, q1 h=PH h=1p0 h=1p1 hq1 hq1 8 The general case was obtained by Diewert (2001, p. 222).

H, p1 1, .

.

.

, p1 1, .

.

.

, p0 18.12 Returning to the inequality (18.7), it can be seen that the theoretical Paasche conditional plutocratic cost of living index, P*( p0 H, u1, e1), is bounded from below by the observable plutocratic or disaggregated Paasche price index, PPP.
Diewert (1983a, p. 191) ﬁrst obtained the inequality (18.7) for the case where the environmental variables are absent from the household utility and cost functions, and prices are equal across households.

The general case is attributable to Diewert (2001, p. 223).
18.13 In the following section, it will be shown how to obtain a theoretical plutocratic cost of living index that is bounded from above and below rather than the theoretical indices in inequalities (18.3) and (18.7) that just have the one-sided bounds.
The Fisher plutocratic price index 18.14 Using the inequalities (18.3) and (18.7) and the continuity properties of the conditional plutocratic cost of living P*( p0 H, p1 H, u, e) deﬁned by equation (18.2), it is possible to modify the method of proof used by Konu¨ s (1924) and Diewert (1983a, p. 191) and establish the following result:9 1, .

.

.

, p0 1, .

.

.

, p1 1, u* h1, e* h2, .

.

.

, e* h and u1 There exists a reference utility vector u* (u* 2, .

.

.
.

.

, u* H) such that the household h reference utility level u* h lies between the household h period 0 and 1 utility levels, u0 h respectively for h=1, .

.

.

, H, and there exist household environmental vectors e* h (e* hM) such that the household h reference mth environmental variable e* hm lies between the household h period 0 and 1 levels for the mth envir- onmental variable, e0 hm and e1 hm respectively for m=1, 2, .

.

.

, M and h=1, .

.

.

, H, and the conditional 1, .

.

.


, p0 H, plutocratic cost of p1 1, .

.

.

, p1 intermediate evaluated at reference utility vector u* and the intermediate refer- ence vector of household environmental variables living index P*( p0 H, u*, e*) this 9 See Diewert (2001, p. 223).

Note that the household cost functions must be continuous in the environmental variables; this is a real restriction on the types of environmental variables which can be accommodated by the result.
339 CONSUMER PRICE INDEX MANUAL: THEORY AND PRACTICE 1, e* 2, .

.

.

, e* e* (e* H) lies between the observable (in principle) plutocratic Laspeyres and Paasche price indices, PPL and PPP, deﬁned above by the last equalities in (18.3) and (18.7).

18.15 The above result says that the theoretical national plutocratic conditional consumer price index P*( p0 H, u*, e*) lies between the pluto- cratic or disaggregated Laspeyres index PPL and the plutocratic or disaggregated Paasche index PPP.
Hence if PPL and PPP are not too diﬀerent, a good point approximation to the theoretical national plutocratic consumer price index will be the plutocratic or dis- aggregated Fisher index PPF deﬁned as: 1, .

.

.

, p0 1, .

.

.

, p1 H, p1 PPF p ﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃ PPLPPP (18:9) The plutocratic Fisher price index PPF is computed just like the usual Fisher price index, except that each commodity in each region (or for each household) is regarded as a separate commodity.
Of course, this index will satisfy the time reversal test.

18.16 Since statistical agencies do not calculate Laspeyres, Paasche and Fisher price indices by taking inner products of price and quantity vectors, as was done in equation (18.9) and the previous deﬁnitions, it will be useful to obtain formulae for the Laspeyres and Paasche indices that depend only on price rela- tives and expenditure shares.

In order to do this, it is necessary to introduce some notation.
Deﬁne the expenditure share of household h on commodity i in period t as St hi hiqt pt hi hkqt pt hk n P k=1 ; t=0, 1; h=1, 2, .

.

.

, H; i=1, 2, .

.

.

, n (18:10) Deﬁne the expenditure share of household h in total period t consumption as: hiqt pt hi n P i=1 n P i=1 St h = hqt pt h H P k=1 t=0, 1; h=1, 2, .

.

.
.

.

, H H P k=1 ikqt pt ik kqt pt k (18:11) Finally, deﬁne the national expenditure share of com- modity i in period t as: st i hiqt pt hi H P h=1 H P k=1 t=0, 1; i=1, 2, .

.

.

, n kqt pt k H = P h=1 hiqt pt hi hqt pt h 0 B B B @ 1 C C C A hqt pt h H P k=1 kqt pt k 340 = H P h=1 H P k=1 hi pt st hqt h kqt pt k H = P h=1 hiSt st h (18:12) The Laspeyres price index for region h (or household h) is deﬁned as: h=1, 2, .

.

.
, H PLh = p1 hq0 h hq0 p0 h p1 n P hi p0 i=1 hi p0 hq0 h p1 hi p0 hi s0 hi n =P i=1 hiq0 p0 hi (18:13) 18.17 Referring back to equation (18.3), the pluto- cratic national Laspeyres price index, PPL, can be rewritten as follows: PPL H P h=1 H P h=1 hq0 p1 h hq0 p0 h (18:14) H = P h=1 p1 hq0 h hq0 p0 h 0 B B B @ hq0 p0 h H P h=1 hq0 p0 h 1 C C C A H = P h=1 S0 h p1 hq0 h hq0 p0 h H = P h=1 S 0 h PLh H X = S 0 h n X h=1 i=1 = H X n X h=1 i=1 h s 0 S 0 hi s 0 hi p1 hi p0 hi p1 hi p0 hi (18:15) (18:16) Equation (18.15) shows that the plutocratic national Laspeyres price index is equal to a (period 0) regional the regional expenditure share-weighted average of Laspeyres price indices.
Equation (18.16) shows that the national Laspeyres price index is equal to a (period 0) expenditure share-weighted average of the regional price relatives, ( p1 hi), where the corresponding weight, h s 0 S 0 is the period 0 national expenditure share of hi, commodity i in region h. hi=p0 18.18 The Paasche price index for region h (or household h) is deﬁned as: PPh hq1 p1 h p0 hq1 h h=1, 2, .

.

.
, H = n P i=1 1 p0 hi p1 hi hiq1 p1 hi=p1 hq1 h THE ECONOMIC APPROACH TO INDEX NUMBER THEORY: THE MANY-HOUSEHOLD CASE 1 1 p1 hi p0 hi = s1 hi n P i=1 ( n = P i=1 ) 1 s1 hi 1 p1 hi p0 hi (18:17) 18.19 Referring back to equation (18.7), the pluto- cratic national Paasche price index, PPP, can be rewrit- ten as follows: where PL is the usual aggregate Laspeyres price index based on the assumption that each household faces the same vector of commodity prices; see equation (18.5) for the deﬁnition of PL.
Under the equal prices across households assumption, the formula for the plutocratic Paasche index becomes: ( H PPP= P h=1 ) 1 n P i=1 hs1 S1 hi 1 p1 hi p0 hi ) 1 using assumptions (18:4) ( H = P h=1 n P i=1 hs1 S1 hi 1 p1 i p0 i ) 1 (18:18) ( n = P i=1 s1 i =PP 1 p1 i p0 i using equation (18:12) for t=1 (18:22) PPP = H P h=1 H P h=1 hq1 p1 h p0 hq1 h 8 >< >: H P h=1 hq1 p0 h hq1 p1 h 1 0 B @ hq1 p1 h PH hq1 p1 h h=1 1 C A 9 >= >; 1 hq0 p1 h hq0 p0 h 1 S1 h = H P h=1 H = P h=1 1 hP 1 S1 Ph (18:19) ( H = P h=1 S1 h n P i=1 s1 hi 1 p1 hi p0 hi ( H = P h=1 n P i=1 hs1 S1 hi 1 p1 hi p0 hi ) 1 ) 1 (18:20) Equation (18.19) shows that the national plutocratic Paasche price index is equal to a (period 1) regional expenditure share-weighted harmonic mean of the regional Paasche price indices.
Equation (18.20) shows that the national Paasche price index is equal to a (period 1) expenditure share-weighted harmonic average of the regional price relatives, ( p1 hi), where the weight for this price relative, S1 hi, is the period 1 national expenditure share of commodity i in region h. hi=p0 hs1 18.20 Of course, the share formulae for the pluto- cratic Paasche and Laspeyres indices, PPP and PPL, given by equations (18.20) and (18.16), can now be used to calculate the plutocratic Fisher index, PPF ½Ppp PPL1=2.
If prices are equal across regions, the formulae (18.16) and (18.20) simplify.
The formula for the plu- tocratic Laspeyres index becomes: 18.21 H PPL= P h=1 n P i=1 hs0 S0 hi H = P h=1 n P i=1 hs0 S0 hi p1 hi p0 hi p1 i p0 i using assumptions (18:4) using equation (18:12) for t=0 s0 i p1 i p0 i n =P i=1 =PL where PP is the usual aggregate Paasche price index based on the assumption that each household faces the same vector of commodity prices; see equation (18.8) for the deﬁnition of PP.
18.22 Thus with the assumption that commodity prices are the same across regions, in order to calculate national Laspeyres and Paasche indices, only ‘‘national’’ price relatives and national commodity expenditure shares are required for the two periods under con- sideration.
If there is regional variation in prices, how- ever, then the simpliﬁed formulae (18.21) and (18.22) are not valid and it is necessary to use the earlier formulae (18.16) and (18.20), which require the use of regional price relatives and regional expenditure shares.

Democratic versus plutocratic cost of living indices 18.23 The plutocratic indices considered above weight each household in the economy according to the size of its expenditures in the two periods under con- sideration.
Instead of weighting in this way, it is possible to deﬁne theoretical indices (and ‘‘practical’’ approx- imations to them) that give each household or house- hold group in the economy an equal weight.

Following Prais (1959), such an index will be called a democratic index.

In this section, the plutocratic index number theory developed in paragraphs 18.3 to 18.22 will be reworked into the democratic framework.
18.24 Making the same assumptions as in paragraph 18.4, deﬁne the class of conditional democratic cost of living indices, P* D( p0, p1, u, e1, e2, .

.

.

, eH), pertaining to periods 0 and 1 for the arbitrary utility vector of household utilities u:(u1, u2, .

.

.

, uH) and for the arbi- trary vectors of household environmental variables eh for h=1, 2, .

.

.

, H as follows: H, p1 1, .

.

.

, p1 H, u, e1, e2, .

.

.

, eH) D( p0 P* 1, .

.

.


, p0 H 1 P H h=1 Ch(uh, eh, p1 h) Ch(uh, eh, p0 h) (18:23) (18:21) Thus P* D is a simple unweighted arithmetic average of the individual household conditional cost of living 341 CONSUMER PRICE INDEX MANUAL: THEORY AND PRACTICE a in indices, Ch(uh, eh, p1 h)=Ch(uh, eh, p0 In the numer- h).

ator and denominator of these conditional indices, only the price variables are diﬀerent, which is precisely what is wanted a theoretical consumer price index.
If the vector of environmental variables, eh, is not present in the cost function of household h, then the conditional index Ch(uh, eh, p1 h)= Ch(uh, eh, p0 an true ordinary Konu¨ s h) cost of living index of the type deﬁned earlier in Chapter 17. deﬁnition becomes of 1, u1 2, .

.

.

e0 2, .

.

.

u1 specialize H).


Further vector of household environmental 18.25 Now specialize the general deﬁnition (18.23) by replacing the general utility vector u by either the period 0 vector of household utilities u0 (u0 1, u0 2, .

.

.

u0 H) or the period 1 vector of household utilities u1 (u1 the general deﬁnition by replacing the general household environ- mental vectors (e1, e2, .

.

.
.

.

eH):e by either the period 0 variables e0 (e0 1, e0 H) or the period 1 vector of household H).

The environmental choice of the base period vector of utility levels and base period environmental variables to the Laspeyres conditional democratic cost of living index, H, u0, e0), while the choice of D( p0 P* the period 1 vector of utility levels and period 1 environmental variables leads to the Paasche condi- D( p0 tional democratic cost of 1, .

.

.
.

.

, p0 H, p1 these two democratic indices satisfy some interesting inequalities, which are derived below.

living index, P* H, u1, e1).

1, .

.

.

, p1 1, .

.

.

, p0 1, .

.

.

, p1 2, .

.

.

; e1 turns out variables e1 (e1 H, p1 leads 1, e1 that It 18.26 Specializing deﬁnition (18.23), the Laspeyres 1, .

.

.

, conditional democratic cost of living index, P* H, u0, e0), may be written as follows: H, p1 p0 1, .

.

.

, p1 D( p0 1, e0 2, .

.

.

, e0 H) D( p0 P* 1, .

.

.


, p0 H 1 P H h=1 1, .

.

.

, p1 H, p1 Ch(u0 h, e0 h, e0 Ch(u0 Ch(u0 h, e0 hq0 p0 h H, u0, e0 h, p1 h) h, p0 h) h, p1 h) 1 H H = P h=1 using equation (18:1) for t=0 p1 hq0 1 h hq0 p0 H h H P h=1 (18:24) h, e0 p1q0 p1 h h is feasible for the cost minimization problem for hq: f h(q, e0 h) minq h) u0 h h, p1 since Ch(u0 and q0 h=1, 2, .

.

.
.

.

, H PDL h=1 p1 1 H where PDL is deﬁned to be the observable (in principle) democratic Laspeyres price index, PH hq0 h, which uses the individual vectors of household or regional quantities for period 0, (q0 H), as quantity weights.

18.27 In a similar manner, specializing deﬁnition (18.23), the Paasche conditional democratic cost of H, u1, e1), may be H, p1 living index, P* written as follows: 1, .

.

.

, p0 1, .

.

.

, q0 1, .

.

.

, p1 D( p0 h=p0 hq0 342 1, e1 2, .

.


.

, e1 H) D( p0 P* H, u1, e1 h, p1 h) h, p0 h) 1, .

.

.

, p0 H 1 P H h=1 1 H H, p1 Ch(u1 Ch(u1 1, .

.

.
.

.

, p1 h, e1 h, e1 hq1 p1 h h, p0 h, e1 Ch(u1 h) H = P h=1 using equation (18:1) for t=1 H P h=1 p1 hq1 1 h p0 hq1 H h PDP using a feasibility argument (18:25) h=1 hq1 hq1 p1 1 H where PDP is deﬁned to be the democratic Paasche price index, PH h=p0 h, which uses the individual vec- tor of household h quantities for period 1, q1 h, as quantity weights for term h in the summation of individual household Paasche indices.
Thus, it can be seen that the theoretical Paasche conditional democratic cost of living index, P* H, u1, e1), is bounded from 1, .

.

.

, p1 below by the observable (in principle) democratic Paasche price index PDP.

Diewert (1983a, p. 191) ﬁrst obtained the inequality (18.25) for the case where the environmental variables are absent from the household utility and cost functions, and prices are equal across households.

1, .

.

.
1, .

.

.

, p0 H, p1 D( p0 18.28 It is now shown how to obtain a theoretical democratic cost of living index that is bounded from above and below by observable indices.

Using the inequalities (18.24) and (18.25) and the continuity prop- erties of the conditional democratic cost of living P*( p0 equation (18.23), it is possible to modify the method of proof used by Konu¨ s (1924) and Diewert (1983a, p. 191) and establish the following result: 1, .

.

.

, p0 1, .

.

.


, p1 H, u, e) deﬁned H, p1 by living h1, e* h and u1 1, .

.

.

, p1 index P* hm and e1 h2, .

.

.

, e* There exists a reference utility vector u* (u* 1, 2, .

.

.

, u* u* H) such that the household h reference uti- lity level u* h lies between the household h period 0 and 1 utility levels, u0 h respectively for h=1, .

.

.
.

.

, H. Also, there exist household environmental vectors e* h (e* hM) such that the household h reference mth environmental variable e* hm lies between the household h period 0 and 1 levels for the mth environmental variable, e0 hm respectively for m=1, 2, .

.

.

, M and h=1, .

.

.

, H. The conditional 1, .

.

.


, democratic cost of p0 H, u*, e*), evaluated at this intermediate H, p1 reference utility vector u* and the intermediate reference vector of household environmental vari- ables e* (e* H), lies between the obser- vable (in principle) democratic Laspeyres and Paasche price indices, PDL and PDP, deﬁned above by the last equalities in (18.24) and (18.25).
18.29 The above result says that the theoretical national democratic conditional consumer price index P* H, u*, e*) lies between the demo- cratic Laspeyres index PDL and the democratic Paasche index PDP.

Hence if PDL and PDP are not too diﬀerent, a good point approximation to the theoretical national democratic consumer price index will be the democratic Fisher index PDF, deﬁned as: 1, .

.

.

, p0 1, .

.

.

, p1 2, .

.

.


, e* H, p1 D( p0 D( p0 1, e* THE ECONOMIC APPROACH TO INDEX NUMBER THEORY: THE MANY-HOUSEHOLD CASE PDF p ﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃ PDLPDP (18:26) The democratic Fisher price index, PDF, will satisfy the time reversal test.

18.30 Again, it will be useful to obtain formulae for the democratic Laspeyres and Paasche indices that depend only on price relatives and expenditure shares.
Using deﬁnition (18.10) for the household h expenditure share on commodity i during period t, st hi, the Laspeyres and Paasche price indices for household h can be written in share form as follows: PLh p1 hq0 h hq0 p0 h n ¼ P i=1 s0 hi p1 hi p0 hi ; h=1, .

.

.

, H (18:27) PPh p1 hq1 h p0 hq1 h ( n = P i=1 s1 hi 1 p1 hi p0 hi ) 1 ; h=1, .

.

.
, H: (18:28) Substituting equation (18.27) into the deﬁnition of the democratic Laspeyres index, PDL, leads to the following share type formula:10 H PDL= P h=1 1 H n P i=1 s0 hi p1 hi p0 hi (18:29) Similarly, substituting equation (18.28) into the deﬁni- tion of the democratic Paasche index, PDP, leads to the following share type formula: H PDL= P h=1 1 H )1 ( n P i=1 s1 hi 1 p1 hi p0 hi (18:30) 18.31 The formula for the democratic Laspeyres index in the previous paragraph simpliﬁes if it can be assumed that each household faces the same vector of prices in each of the two periods under consideration.
Under this con- dition, equation (18.28) can be rewritten as follows: n PDL=P s0 di p1 i p0 i (18:31) i=1 where the period 0 democratic expenditure share for commodity i, s0 di, is deﬁned as follows: H i=1, .

.

.

, n (18:32) di P s0 h=1 1 H s0 hi; Thus s0 is simply the arithmetic average (over all di households) of the individual household expenditure shares on commodity i during period 0.
The formula for the democratic Paasche index does not simplify in the same way, under the assumption that households face the same prices in each period, because of the harmonic form of averaging in equation (18.30).
10 Comparing the formula for the democratic Laspeyres index, PDL, with the previous formula (18.16) for the plutocratic Laspeyres index, PPL, it can be seen that the plutocratic weight for the ith price relative for household h is S0 hi, whereas the corresponding democratic weight is (1=H)s0 hi.

Thus households that have larger base period expenditures and hence bigger expenditure shares S0 h get a larger weight in the plutocratic index as compared to the democratic index.
hs0 18.32 The conclusion at this point is that democratic and plutocratic Laspeyres, Paasche and Fisher indices can be constructed by a statistical agency provided that information on household-speciﬁc price relatives, p1 hi=p0 hi, and expenditures is available for both periods under consideration.

If expenditure information is available only for the base period, then only the Laspeyres democratic and plutocratic indices can be constructed.
hs1 18.33 It is now necessary to discuss a practical that problem that statistical agencies face: namely, existing household consumer expenditure surveys, which are used in order to form estimates of household expenditure shares, are not very accurate.

Thus the detailed commodity by region expenditure shares, S0 hs0 hn and S1 hn, which appear in the formulae for the pluto- cratic Laspeyres and Paasche indices, are generally measured with very large errors.
Similarly, the indivi- dual household expenditure shares for the two periods hn and s1 under consideration, s0 hn, which are required in order to calculate the democratic Laspeyres and Paasche indices deﬁned by equations (18.29) and (18.30) respec- tively, are also generally measured with substantial errors.

Hence, it may lead to less overall error if the regional commodity expenditure shares st hn are replaced by the national commodity expenditure shares st n deﬁned by equation (18.12).
Whether this approxima- tion is justiﬁed would depend on a detailed analysis of the situation facing the statistical agency.

In general, complete and accurate information on household expenditure shares will not be available to the statistical agency, and hence statistical estimation and smoothing techniques will have to be used in order to obtain expenditure weights that will be used to weight the price relatives collected by the agency.
18.34 It should be noted that the conditional index framework used above can be used to model situations where household preferences change (continuously) from the base period to the current period: simply choose the environmental variable to be time t. The theoretical results in paragraphs 18.14 and 18.28 imply the existence of cost of living indices that lie between observable Las- peyres and Paasche bounds, where the preference func- tions for the households are taken to be some preferences that are intermediate between the preferences pertaining to the two periods under consideration.
As usual, if the observable bounds are not too far apart, taking the geo- metric average of the bounds leads to an adequate approximation to these theoretical cost of living indices.11 18.35 For criticisms and some limitations of the economic approach to index number theory, see Turvey (2000) and Diewert (2001).12 11 For a more extensive treatment of cost of living theory in the context of taste change, see Balk (1989a).

12 For a vigorous defence of the economic approach, see Triplett (2001).
343 PRICE INDICES USING AN ARTIFICIAL DATA SET 19 Introduction 19.1 In order to give the reader some idea of how much the various index numbers might diﬀer using a ‘‘real’’ data set, virtually all the major indices deﬁned in the previous chapters are computed in this chapter using an artiﬁcial data set consisting of prices and quantities for six commodities over ﬁve periods.

The data are described in paragraphs 19.3 and 19.4.

19.2 The contents of the remaining sections are outlined in this paragraph.
In the section starting with paragraph 19.5, two of the early unweighted indices are computed: the Carli and Jevons indices.

Two of the earliest weighted indices are also computed in this sec- tion: the Laspeyres and Paasche indices.

Both ﬁxed base and chained indices are computed.

In the section start- ing with paragraph 19.9, various asymmetrically1 weighted indices are computed.

In the section starting with paragraph 19.17, symmetrically2 weighted indices are computed.
Some of these indices are superlative, while others are not.

The section starting with para- graph 19.23 computes some superlative indices using two stages of aggregation and compares the resulting two-stage indices with their single-stage counterparts.

The following section computes various Lloyd–Moulton indices3 and compares them with superlative indices.
The section starting with paragraph 19.32 computes two additive percentage change decompositions for the Fisher ideal index and compares the resulting decom- positions, which are found to be very similar.

Up to this point, all the indices that are computed are weighted or unweighted bilateral price indices; i.e., the index number formula depends only on the price and quantity data pertaining to the two periods whose prices are being compared.
In the ﬁnal three sections of this chapter, various indices involving the data pertaining to three or more periods are computed.

In the section starting with paragraph 19.37, Lowe and Young indices are com- puted where the data of period 1 are used as quantity or share weights in conjunction with the price data of periods 3 to 5, so that the weight reference period is 1 and the price reference period is 3.
In the ﬁnal two sections, various mid-year indices are computed that are 1 ‘‘Asymmetric weights’’ means that the quantity or value weights for the prices come from only one of the two periods being compared.

2 ‘‘Symmetric weights’’ means that the quantity or value weights for the prices enter the index number formula in a symmetric or even-handed way.
3 Recall from Chapter 17 that there is a separate Lloyd–Moulton index for each estimated elasticity of substitution parameter s that is inserted into the formula.

based on the Lowe and Young formulae.

Recall that for these two index number formulae, the price reference period does not coincide with the weight reference per- iod.

Thus these indices are not bilateral index number formulae.

The artiﬁcial data set iqt iqt i=1pt i=ptqt, in Table 19.3.
19.3 The period can be thought of as somewhere between a year and ﬁve years.

The trends in the data are generally more pronounced than would be seen in the course of a year.

The price and quantity data are listed in Tables 19.1 and 19.2.

For convenience, the period t nominal expenditures, ptqt Pn i, are listed along with the corresponding period t expenditure shares, i pt st 19.4 The trends that were built into Tables 19.1 to 19.3 are now explained in this paragraph.
Think of the ﬁrst four commodities as the consumption of various classes of goods in some economy, while the last two commodities are the consumption of two classes of ser- vices.

Think of the ﬁrst good as agricultural consumption; its quantity ﬂuctuates around 1 and its price also ﬂuc- tuates around 1.4 The second good is energy consump- tion; its quantity shows a gently upward trend during the ﬁve periods with some minor ﬂuctuations.
Note, how- ever, that the price of energy ﬂuctuates wildly from period to period.5 The third good is traditional manu- factures.

Rather high rates of price inﬂation are assumed for this commodity for periods 2 and 3 which diminish to a very low inﬂation rate by the end of the sample period.6 The consumption of traditional manufactured goods is more or less static in the data set.

The fourth commodity is high-technology manufactured goods, for example computers, video cameras and compact disks.
The demand for these high-technology commodities grows 12 times over the sample period, while the ﬁnal period price is only one-tenth of the ﬁrst period price.

The ﬁfth commodity is traditional services.

The price trends for this commodity are similar to those of tradi- tional manufactures, except that the period-to-period 4 Note, however, that the expenditure share of agricultural products shows a downward trend over time as the economy develops and shifts into services.
5 This is an example of the price bouncing phenomenon noted by Szulc (1983).

Note that the ﬂuctuations in the price of energy that are built into the data set are not that unrealistic: in the past four years, the price of a barrel of crude oil has ﬂuctuated in the range US$12 to US$40.

6 This corresponds roughly to the experience of most industrialized countries over the period from 1973 to the mid-1990s.

Thus, roughly ﬁve years of price movement are compressed into one of our periods.
345 CONSUMER PRICE INDEX MANUAL: THEORY AND PRACTICE Table 19.1 Prices for six commodities Period t pt 1 1.0 1.2 1.0 0.8 1.0 pt 2 1.0 3.0 1.0 0.5 1.0 pt 3 1.0 1.3 1.5 1.6 1.6 pt 4 1.0 0.7 0.5 0.3 0.1 Table 19.2 Quantities for six commodities Period t qt 1 1.0 0.8 1.0 1.2 0.9 qt 2 1.0 0.9 1.1 1.2 1.2 qt 3 2.0 1.9 1.8 1.9 2.0 qt 4 1.0 1.3 3.0 6.0 12.0 pt 5 1.0 1.4 1.7 1.9 2.0 qt 5 4.5 4.7 5.0 5.6 6.5 pt 6 1.0 0.8 0.6 0.4 0.2 qt 6 0.5 0.6 0.8 1.3 2.5 Table 19.3 Expenditures and expenditure shares for six commodities Period t pt qt st 1 st 2 st 3 st 4 st 5 st inﬂation rates are a little higher.
The demand for tradi- tional services, however, grows much more strongly than for traditional manufactures.

The ﬁnal commodity is high-technology services, for example telecommunica- tions, wireless phones, Internet services and stock mar- ket trading.

For this ﬁnal commodity, the price shows a very strong downward trend to end up at 20 per cent of the starting level, while demand increases ﬁvefold.
The movements of prices and quantities in this artiﬁcial data set are more pronounced than the year-to-year move- ments that would be encountered in a typical country, but they do illustrate the problem facing compilers of the consumer price index (CPI); namely, year-to-year price and quantity movements are far from being proportional across commodities, so the choice of index number for- mula will matter.
Early price indices: The Carli, Jevons, Laspeyres and Paasche indices 19.5 Every price statistician is familiar with the Las- peyres index PL deﬁned by equation (15.5) and the Paasche index PP deﬁned by equation (15.6) in Chapter 15.

These indices are listed in Table 19.4 along with two unweighted indices that were considered in previous chapters: the Carli index deﬁned by equation (16.45) and the Jevons index deﬁned by equation (16.47) in Chapter 16.
The indices in Table 19.4 compare the prices in period t with the prices in period 1, that is, they are ﬁxed base indices.

Thus the period t entry for the Carli index, PC, is simply the arithmetic mean of the six price relatives, P6 i=1(1=6) ( pt i ), while the period t entry for the Jevons index, PJ , i=p1 346 is Q6 i=p1 i )1=6.
the six price relatives, the geometric mean of i=1( pt 19.6 Note that by period 5, the spread between the ﬁxed base Laspeyres and Paasche price indices is enor- mous: PL is equal to 1.4400 while PP is 0.7968, a spread of 81 per cent.

Since both these indices have exactly the same theoretical justiﬁcation, it can be seen that the choice of index number formula matters a lot.
The period 5 entry for the Carli index, 0.9833, falls between the corresponding Paasche and Laspeyres indices but the period 5 Jevons index, 0.6324, does not.

Note that the Jevons index is always considerably below the cor- responding Carli index.
This will always be the case (unless prices are proportional in the two periods under consideration) because a geometric mean is always equal to or less than the corresponding arithmetic mean.7 19.7 It is of interest to recalculate the four indices listed in Table 19.4 using the chain principle rather than the ﬁxed base principle.

The expectation is that the spread between the Paasche and Laspeyres indices will be reduced by using the chain principle.

These chain indices are listed in Table 19.5.


19.8 It can be seen comparing Tables 19.4 and 19.5 that chaining eliminated about two-thirds of the spread between the Paasche and Laspeyres indices.

Never- theless, even the chained Paasche and Laspeyres indices diﬀer by about 18 per cent in period 5, so the choice of index number formula still matters.

Note that chaining did not aﬀect the Jevons index.


This is an advantage of the index but the lack of weighting is a fatal ﬂaw.8 Using the economic approach to index number theory, there is an expectation that the ‘‘truth’’ lies between the Paasche and Laspeyres indices.

From Table 19.5, it can be seen that the unweighted Jevons index is far below this acceptable range.
Note that chaining did not aﬀect the Carli index in a systematic way for the artiﬁcial data set: in periods 3 and 4, the chained Carli is above the corresponding ﬁxed base Carli; but in period 5, the chained Carli is below the ﬁxed base Carli.9 Asymmetrically weighted price indices 19.9 This section contains a systematic comparison of all of the asymmetrically weighted price indices (with 7 According to the theorem of the arithmetic and geometric mean; see Hardy, Littlewood and Po´ lya (1934, p. 17).
8 The problem with the evenly weighted geometric mean is that the price declines in high-technology goods and services are given the same weighting as the price changes in the other four commodities (which have rising or stationary price changes), but the expenditure shares of the high-technology commodities remain rather small throughout the ﬁve periods.

Thus weighted price indices do not show the rate of overall price decrease that the unweighted Jevons index shows.
These somewhat negative comments on the use of the unweighted geometric mean as an index number formula at higher levels of aggregation do not preclude its use at the very lowest level of aggregation, where a strong axiomatic justiﬁcation for the use of this formula can be given.

If probability sampling is used at the lowest level of aggregation, then the unweighted geometric mean essentially becomes the logarithmic Laspeyres index.
9 For many data sets, the chained Carli can be expected to be above the corresponding ﬁxed base Carli; see Szulc (1983).
PRICE INDICES USING AN ARTIFICIAL DATA SET Table 19.4 The ﬁxed base Laspeyres, Paasche, Carli and Jevons indices Table 19.6 Asymmetrically weighted ﬁxed base indices Period t P PAL P L P GP P GL P P P HL Period t P L 1.0000 1.4200 1.3450 1.3550 1.4400 P P 1.0000 1.3823 1.2031 1.0209 0.7968 P C 1.0000 1.4000 1.0500 0.9167 0.9833 P J Table 19.5 Chain Laspeyres, Paasche, Carli and Jevons indices Table 19.7 Asymmetrically weighted indices using the chain principle Period t P PAL P L P GP P GL P P P HL Period t P L 1.0000 1.4200 1.3646 1.3351 1.3306 P P 1.0000 1.3823 1.2740 1.2060 1.1234 P C 1.0000 1.4000 1.1664 0.9236 0.9446 P J the exception of the Lloyd–Moulton index, which will be considered later).
The ﬁxed base indices are listed in Table 19.6.

The ﬁxed base Laspeyres and Paasche indices, PL and PP, are the same as those indices listed in Table 19.4.

The Palgrave index, PPAL, is deﬁned by equation (16.55).

The indices denoted by PGL and PGP are the geometric Laspeyres and geometric Paasche indices,10 which are special cases of the class of geometric indices deﬁned by Konu¨ s and Byushgens (1926); see equation (15.78).
For the geometric Laspeyres index, PGL, the exponent weight ai for the ith price relative is s1 i , where s1 i is the base period expenditure share for commodity i.

The resulting index should be considered an alternative to the ﬁxed base Laspeyres index, since both of these indices make use of the same information set.

For the geometric Paasche index, PGP, the exponent weight for the ith price relative is st i is the current period expenditure shares.
Finally, the index PHL is the harmonic Laspeyres index that was deﬁned by equation (16.59).

i, where st 19.10 By looking at the period 5 entries in Table 19.6, it can be seen that the spread between all these ﬁxed base asymmetrically weighted indices has increased to be even larger than the earlier spread of 81 per cent between the ﬁxed base Paasche and Laspeyres indices.

In Table 19.6, the period 5 Palgrave index is about three times as big as the period 5 harmonic Laspeyres index, PHL.
Again, this illustrates the point that because of the non-proportional growth of prices and quantities in most economies today, the choice of index number formula is very important.

19.11 It is possible to explain why certain of the indices in Table 19.6 are bigger than others.
It can be shown that a weighted arithmetic mean of n numbers is equal to or greater than the corresponding weighted geometric mean of the same n numbers, which in turn is equal to or greater than the corresponding weighted harmonic mean of the same n numbers.11 It can be seen that the three indices PPAL, PGP and PP all use the current period expenditure shares st to weight i the price relatives ( pt i ), but PPAL is a weighted arithmetic mean of these price relatives, PGP is a weighted geometric mean of these price relatives and PP is a weighted harmonic mean of these price rela- tives.
Thus by Schlo¨ milch’s inequality, it must be the case that:12 i=p1 PPAL PGP PP (19:1) i to weight the price relatives ( pt 19.12 Table 19.6 shows that the inequalities (19.1) hold for each period.
It can also be veriﬁed that the three indices PL, PGL and PHL all use the base period expen- diture shares s1 i ), but PL is a weighted arithmetic mean of these price relatives, PGL is a weighted geometric mean of these price rela- tives, and PHL is a weighted harmonic mean of these price relatives.

Thus by Schlo¨ milch’s inequality, it must be the case that:13 i=p1 PL PGL PHL (19:2) Table 19.6 shows that the inequalities (19.2) hold for each period.
19.13 All the asymmetrically weighted price indices are compared using the chain principle and are listed in Table 19.7.
19.14 Table 19.7 shows that although the use of the chain principle dramatically reduced the spread between the Paasche and Laspeyres indices PP and PL compared to the corresponding ﬁxed base entries in Table 19.6, the spread between the highest and lowest asymmetrically weighted indices in period 5 (the Palgrave index PPAL and the harmonic Laspeyres index PHL) does not fall as much: the ﬁxed base spread is 1.6720/0.5556=3.01, while the corresponding chain spread is 1.7893/0.7299=2.45.
10 Vartia (1978, p. 272) used the terms logarithmic Laspeyres and logarithmic Paasche, respectively.

11 This follows from Schlo¨ milch’s (1858) inequality; see Hardy, Lit- tlewood and Po´ lya (1934, p. 26).

12 These inequalities were noted by Fisher (1922, p. 92) and Vartia (1978, p. 278).

13 These inequalities were also noted by Fisher (1922, p. 92) and Vartia (1978, p. 278).
347 CONSUMER PRICE INDEX MANUAL: THEORY AND PRACTICE Thus, in this particular case, the use of the chain prin- ciple combined with the use of an index number formula that uses the weights of only one of the two periods being compared did not lead to a signiﬁcant narrowing of the huge diﬀerences that these formulae generated using the ﬁxed base principle.

With respect to the Paasche and Laspeyres formulae, however, chaining did signiﬁcantly reduce the spread between these two indices.
19.15 Is there an explanation for the results reported in the previous paragraph?

It can be shown that all six of the indices that are found in the inequalities (19.1) and (19.2) approximate each other to the ﬁrst order around an equal prices and quantities point.

Thus with smooth trends in the data, it is expected that all the chain indices will more closely approximate each other than the ﬁxed base indices because the changes in the individual prices and quantities are smaller using the chain principle.
This expectation is realized in the case of the Paasche and Laspeyres indices, but not with the others.

For some of the commodities in the data set, however, the trends in the prices and quantities are not smooth.

In particu- lar, the prices for the ﬁrst two commodities (agricultural products and oil) bounce up and down.

As noted by Szulc (1983), this will tend to cause the chain indices to have a wider dispersion than their ﬁxed base counter- parts.
In order to determine if it is the bouncing prices problem that is causing some of the chained indices in Table 19.7 to diverge from their ﬁxed base counterparts, all the indices in Tables 19.6 and 19.7 were computed again but excluding commodities 1 and 2 from the computations.

The results of excluding these bouncing commodities may be found in Tables 19.8 and 19.9.
19.16 It can be seen that excluding the bouncing price commodities does cause the chain indices to have a much narrower spread than their ﬁxed base counter- parts.

Thus, the conclusion is that if the underlying price and quantity data are subject to reasonably smooth trends over time, then the use of chain indices will narrow considerably the dispersion in the asymme- trically weighted indices.
In the next section, index number formulae that use weights from both periods in a symmetric or even-handed manner are computed.
Table 19.8 Asymmetrically weighted ﬁxed base indices for commodities 3 – 6 Period t P PAL P L P GP P GL P P P HL Table 19.9 Asymmetrically weighted chained indices for commodities 3 – 6 Period t P PAL P L P GP P GL P P P HL Symmetrically weighted indices: Superlative and other indices 19.17 Symmetrically weighted indices can be decom- posed into two classes: superlative indices and other symmetrically weighted indices.
Superlative indices have a close connection to economic theory.

As was seen in paragraphs 17.27 to 17.49 of Chapter 17, a superlative index is exact for a representation of the consumer’s preference function or the dual unit cost function that can provide a second-order approximation to arbitrary (homothetic) preferences.
Four important superlative indices were considered in previous chapters: the Fisher ideal price index PF , deﬁned by equation (15.12); the Walsh price index PW , deﬁned by equation (15.19) (this price index also corresponds to the quantity index Q1, deﬁned by equation (17.33) in Chapter 17); the T€oornqvist–Theil price index PT , deﬁned by equa- tion (15.81); the implicit Walsh price index PIW that corresponds to the Walsh quantity index QW deﬁned in Chapter 15 (this is also the index P1 deﬁned by equation (17.38)).
19.18 These four symmetrically weighted superlative price indices are listed in Table 19.10 using the ﬁxed base principle.

Also listed in Table 19.10 are two symme- trically weighted (but not superlative) price indices:14 the Marshall–Edgeworth price index PME, deﬁned in paragraph 15.18; the Drobisch price index PD, deﬁned by equation (15.12).

19.19 Note that the Drobisch index PD is always equal to or greater than the corresponding Fisher index PF .
This follows from the fact that the Fisher index is the geometric mean of the Paasche and Laspeyres indi- ces, while the Drobisch index is the arithmetic mean of the Paasche and Laspeyres indices, and an arithmetic mean is always equal to or greater than the corre- sponding geometric mean.
Comparing the ﬁxed base asymmetrically weighted indices in Table 19.6 with the symmetrically weighted indices in Table 19.10, it can be seen that the spread between the lowest and highest index in period 5 is much less for the symmetrically weighted indices.

The spread is 1.6720/0.5556=3.01 for the asymmetrically weighted indices, but only 1.2477/ 0.9801=1.27 for the symmetrically weighted indices.
If the comparisons are restricted to the superlative indices listed for period 5 in Table 19.10, then this spread is further reduced to 1.2477/1.0712=1.16; i.e., the spread between the ﬁxed base superlative indices is ‘‘only’’ 16 per cent compared to the ﬁxed base spread between the Paasche and Laspeyres indices of 81 per cent (1.4400/ 0.7968=1.81).

There is an expectation that the spread between the superlative indices will be further reduced by using the chain principle.
14 Diewert (1978, p. 897) showed that the Drobisch–Sidgwick–Bowley price index approximates any superlative index to the second order around an equal price and quantity point; i.e., PSB is a pseudo-super- lative index.

Straightforward computations show that the Marshall– Edgeworth index PME is also pseudo-superlative.
Table 19.10 Symmetrically weighted ﬁxed base indices Period t PT PIW PW PF PD PME Table 19.11 Symmetrically weighted indices using the chain principle Period t P T P IW P W P F P D P ME The symmetrically weighted indices are com- puted using the chain principle.
The results may be found in Table 19.11.

19.21 A quick glance at Table 19.11 shows that the combined eﬀect of using both the chain principle as well as symmetrically weighted indices is to dramatically reduce the spread between all indices constructed using these two principles.
The spread between all the symmetrically weighted indices in period 5 is only 1.2333/1.2155=1.015 or 1.5 per cent and the spread between the four super- lative indices in period 5 is an even smaller 1.2333/ 1.2224=1.009, or about 0.1 per cent.
The spread in period 5 between the two most commonly used superlative indices, the Fisher PF and the To¨ rnqvist PT , is truly tiny: 1.2226/1.2224=0.0002.15 19.22 The results listed in Table 19.11 reinforce the numerical results tabled by Hill (2002) and Diewert (1978, p. 894); the most commonly used chained super- lative indices will generally give approximately the same numerical results.16 In particular, the chained Fisher, To¨ rnqvist and Walsh indices will generally approximate each other very closely.
Superlative indices constructed in two stages of aggregation 19.23 Attention is now directed to the diﬀerences between superlative indices and their counterparts that are constructed in two stages of aggregation; see para- graphs 17.55 to 17.60 of Chapter 17 for a discussion of the issues and a listing of the formulae used.

Using the artiﬁcial data set, the ﬁrst four commodities are com- bined into a goods aggregate and the last two com- modities into a services aggregate.
In the second stage of PRICE INDICES USING AN ARTIFICIAL DATA SET aggregation, the goods and services components will be aggregated into an all-items index.

19.24 The results for the two-stage aggregation procedure using period 1 as the ﬁxed base for the Fisher index PF , the To¨ rnqvist index PT and the Walsh and implicit Walsh indexes, PW and PIW , are reported in Table 19.12.
19.25 Table 19.12 shows that the ﬁxed base single stage superlative indices generally approximate their ﬁxed base two-stage counterparts fairly closely, with the exception of the Fisher formula.

The divergence between the single-stage Fisher index PF and its two-stage coun- terpart PF2S in period 5 is 1.1286/1.0712=1.05 or 5 per cent.

The other divergences are 2 per cent or less.

19.26 Using chain indices, the results of the two- stage aggregation procedure are reported in Table 19.13.
Again, the single-stage and their two-stage counter- parts are listed for the Fisher index PF , the To¨ rnqvist index PT and the Walsh and implicit Walsh indexes, PW and PIW .

19.27 Table 19.13 shows that the chained single- stage superlative indices generally approximate their ﬁxed base two-stage counterparts very closely indeed.

chained single-stage The divergence between the To¨ rnqvist index PT and its two-stage counterpart PT2S in period 5 is 1.2300/1.2224=1.006 or 0.6 per cent.
The other divergences are all less than this.

Given the large dispersion in period-to-period price movements, these two-stage aggregation errors are not large.

Lloyd – Moulton price indices 19.28 The next formula that will be illustrated using the artiﬁcial data set is the Lloyd (1975) and Moulton (1996) index PLM, deﬁned by equation (17.71).

Recall that this formula requires an estimate for the parameter s, the elasticity of substitution between all commodities being aggregated.
Recall also that if s equals 0, then the Lloyd–Moulton index collapses down to the ordinary Laspeyres index, PL.

When s equals 1, the Lloyd– Moulton index is not deﬁned, but it can be shown that the limit of PLMs as s approaches 1 is PGL, the geometric Laspeyres index or the logarithmic Laspeyres index with base period shares as weights.

This index uses the same basic information as the ﬁxed base Laspeyres index PL, and so it is a possible alternative index for CPI compilers to use.
As was shown by Shapiro and Wilcox (1997a),17 the Lloyd–Moulton index may be used to approximate a superlative index using the same information that is used in the construction of a ﬁxed base Laspeyres index, provided that an estimate for the parameter s is avail- able.

This methodology will be tested using the artiﬁcial data set.

The superlative index that is to be approximated is the chain Fisher index18 (which approximates the other 15 In other periods, the diﬀerences were nevertheless larger.
On average over the last four periods, the chain Fisher and the chain To¨ rnqvist indices diﬀered by 0.0025 percentage points.

16 More precisely, the superlative quadratic mean of order r price indices Pr deﬁned by equation (17.35) and the implicit quadratic mean of order r price indices Pr* deﬁned by equation (17.32) will generally closely approximate each other, provided that r is in the interval 0 r 2.
17 Alterman, Diewert and Feenstra (1999) also used this methodology in the context of estimating superlative international trade price indices.

18 Since there is still a considerable amount of dispersion among the ﬁxed base superlative indices and practically no dispersion between the chained superlative indices, the Fisher chain index is taken as the target index rather than any of the ﬁxed base superlative indices.
349 CONSUMER PRICE INDEX MANUAL: THEORY AND PRACTICE Table 19.12 Fixed base superlative single-stage and two-stage indices Period t P F 1.0000 1.4011 1.2721 1.1762 1.0712 P F 2S 1.0000 1.4004 1.2789 1.2019 1.1286 P T 1.0000 1.4052 1.2890 1.2268 1.2477 P T 2S 1.0000 1.4052 1.2872 1.2243 1.2441 Table 19.13 Chained superlative single-stage and two-stage indices Period t P F 1.0000 1.4011 1.3185 1.2689 1.2226 P F 2S 1.0000 1.4004 1.3200 1.2716 1.2267 P T 1.0000 1.4052 1.3112 1.2624 1.2224 P T 2S 1.0000 1.4052 1.3168 1.2683 1.2300 P W 1.0000 1.4017 1.2850 1.2193 1.1850 P W 1.0000 1.4017 1.3207 1.2731 1.2304 P W 2S 1.0000 1.4015 1.2868 1.2253 1.2075 P W 2S 1.0000 1.4015 1.3202 1.2728 1.2313 P IW 1.0000 1.4015 1.2854 1.2174 1.2206 P IW 1.0000 1.4015 1.3203 1.2723 1.2333 Table 19.14 Chained Fisher and ﬁxed base Lloyd – Moulton indices Period t P F 1.0000 1.4011 1.3185 1.2689 1.2226 P LM0 1.0000 1.4200 1.3450 1.3550 1.4400 P LM:2 1.0000 1.4005 1.3287 1.3172 1.3940 P LM:3 1.0000 1.3910 1.3201 1.2970 1.3678 P LM:4 1.0000 1.3818 1.3113 1.2759 1.3389 P LM:5 1.0000 1.3727 1.3021 1.2540 1.3073 P LM:6 1.0000 1.3638 1.2927 1.2312 1.2726 P LM:7 1.0000 1.3551 1.2831 1.2077 1.2346 P LM:8 1.0000 1.3466 1.2731 1.1835 1.1932 P IW 2S 1.0000 1.4022 1.2862 1.2209 1.2240 P IW 2S 1.0000 1.4022 1.3201 1.2720 1.2330 P LM1 1.0000 1.3300 1.2523 1.1331 1.0999 chained superlative indices listed in Table 19.11 very closely).
The chained Fisher index PF is listed in column 2 of Table 19.14 along with the ﬁxed base Lloyd– Moulton indices PLMs for s equal to 0 (this reduces to the ﬁxed base Laspeyres index PL), 0.2, 0.3, 0.4, 0.5, 0.6, 0.7, 0.8 and 1 (which is the ﬁxed base geometric index PGL).
Note that the Lloyd–Moulton indices steadily decrease as the elasticity of substitution s is increased.19 19.29 Table 19.14 shows that no single choice of the elasticity of substitution s will lead to a Lloyd–Moulton price index PLMs that will closely approximate the chained Fisher index PF for periods 2, 3, 4 and 5.
To approximate PF in period 2, it is necessary to choose s close to 0.1; to approximate PF in period 3, choose s close to 0.3; to approximate PF in period 4, choose s between 0.4 and 0.5; and to approximate PF in period 5, choose s between 0.7 and 0.8.20 19.30 The computations for the Lloyd–Moulton indices listed in Table 19.14 are now repeated except that the chain principle is used to construct the Lloyd– Moulton indices; see Table 19.15.
Again, the object is to approximate the chained Fisher price index PF which is listed as the second column in Table 19.15.

In Table 19.15, PLM0 is the chained Laspeyres index and PLM1 is the chained geometric Laspeyres or geometric index using the expenditure shares of the previous period as weights.
19.31 Table 19.15 shows that again no single choice of the elasticity of substitution s will lead to a Lloyd– Moulton price index PLMs that will closely approxi- mate the chained Fisher index PF for all periods.

To approximate PF in period 2, choose s close to 0.1; to approximate PF in period 3, choose s close to 0.2; to approximate PF in period 4, choose s between 0.2 and 0.3; and to approximate PF in period 5, choose s between 0.3 and 0.4.
It should be noted, however, that if s is chosen to equal to 0.3 and the resulting chained Lloyd–Moulton index PLM.3 is used to approximate the chained Fisher index PF, then a much better approx- imation to PF results than that provided by either the chained Laspeyres index (see PLM0 in the third column of Table 19.15) or the ﬁxed base Laspeyres index (see PLM0 in the third column of Table 19.14).21 The tenta- tive conclusions on the use of the Lloyd–Moulton index to approximate superlative indices that can be drawn from the above tables are: the elasticity of substitution parameter s which appears in the Lloyd–Moulton formula is unlikely to remain constant over time, and hence it will be necessary for statistical agencies to update their esti- mates of s at regular intervals; the use of the Lloyd–Moulton index as a real-time preliminary estimator for a chained superlative index 19 This follows from Schlo¨ milch’s (1858) inequality again.
20 Unfortunately, for this data set, neither the ﬁxed base Laspeyres index PL=PLM0 nor the ﬁxed base weighted geometric index PGL=PLM1 is very close to the chain Fisher index for all periods.

For less extreme data sets, the ﬁxed base Laspeyres and ﬁxed base geo- metric indices will be closer to the chained Fisher index.
21 For this particular data set, the ﬁxed base or chained geometric indices using either the expenditure weights of period 1 (see the last column of Table 19.14) or using the weights of the previous period (see the last column of Table 19.15) do not approximate the chained Fisher index very closely.

For less extreme data sets, however, the use of chained Laspeyres or geometric indices may approximate a chained superlative index adequately.
350 PRICE INDICES USING AN ARTIFICIAL DATA SET Table 19.15 Chained Fisher and chained Lloyd – Moulton indices Period t P F P LM0 P LM:2 P LM:3 P LM:4 P LM:5 P LM:6 P LM:7 P LM:8 P LM1 Table 19.16 Diewert’s additive percentage change decomposition of the Fisher index Period t P F 0.4011 0.0589 0.0376 0.0365 v F 1Dp1 0.0176 0.0118 0.0131 0.0112 v F 2Dp2 0.1877 0.1315 0.0345 0.0316 v F 3Dp3 0.0580 0.0246 0.0111 0.0000 v F 4Dp4 0.0351 0.0274 0.0523 0.0915 v F 5Dp5 0.1840 0.0963 0.0635 0.0316 v F 6Dp6 0.0111 0.0092 0.0123 0.0194 seems warranted, provided that the statistical agency can provide estimates for chained superlative indices on a delayed basis.
The Lloyd–Moulton index would provide a useful supplement to the traditional ﬁxed base Laspeyres price index.
Additive percentage change decompositions for the Fisher ideal index i to pt i pt1 i 19.32 The next formulae to be illustrated using the artiﬁcial data set are the additive percentage change decompositions for the Fisher ideal index, discussed in paragraphs 16.62 to 16.73 of Chapter 16.22 The chain links for the Fisher price index are ﬁrst decomposed into additive components using the formulae (16.38) to (16.40).

The results of the decomposition are listed in Table 19.16.
Thus PF 1 is the percentage change in the Fisher ideal chain link going from period t 1 to t, and the decomposition factor vFiDPi=vFi(Pt ) is the contribution to the total percentage change of the change in the ith price from pt1 i for i=1, 2, .

.

.

, 6.
.

.

, 6.

19.33 Table 19.16 shows that the price index going from period 1 to 2 grew about 40 per cent, and the major contributors to this change were the increases in the price of commodity 2, energy (18.77 per cent), and in commodity 5, traditional services (18.4 per cent).

The increase in the price of traditional manufactured goods, commodity 3, contributed 5.8 per cent to the overall increase of 40.11 per cent.
The decreases in the prices of high-technology goods (commodity 4) and high- technology services (commodity 6) oﬀset the other in- creases by 3.51 per cent and 1.11 per cent going from period 1 to 2.

Going from period 2 to 3, the overall change in prices was negative: 5.89 per cent.

The reader can read across row 3 of Table 19.16 to see what was the contribution of the six component price changes to the overall price change.
It is evident that a big price change in a particular component i, combined with a big expen- diture share in the two periods under consideration will lead to a big decomposition factor, vFi.
19.34 The next set of computations to be illustrated using the artiﬁcial data set is the additive percentage change decomposition for the Fisher ideal index accord- ing to Van Ijzeren (1987, p. 6), which was mentioned in footnote 43 of Chapter 16.23 The price counterpart to the additive decomposition for a quantity index is: PF ( p0, p1, q0, q1)= n P i=1 n P i=1 Fi p1 q* i Fi p0 q* i (19:3) where the reference quantities need to be deﬁned somehow.
Van Ijzeren (1987, p. 6) showed that the fol- lowing reference weights provide an exact additive representation for the Fisher ideal price index: Fi (1=2)q0 q* for i=1, 2, .

.

.

, 6 i +f(1=2)q1 i =QF ( p0, p1, q0, q1)g (19:4) where QF is the overall Fisher quantity index.
Thus using the Van Ijzeren quantity weights qFi* , the following Van Ijzeren additive percentage change decomposition for the Fisher price index is obtained: PF ( p0, p1, q0, q1) 1= P 6 i=1 Fi p1 q* i 6 P i=1 1 Fi p0 q* i = 6 X i=1 Fif p1 v* i p0 i g (19:5) where the Van Ijzeren weight for commodity i, vFi* , is deﬁned as Fi q* v* Fi 6 P i=1 Fi p0 q* i for i=1, 2, .

.

.

, 6 (19:6) 22 See Diewert (2002a, p. 73).


23 See Reinsdorf, Diewert and Ehemann (2002) for additional infor- mation on this decomposition.
351 CONSUMER PRICE INDEX MANUAL: THEORY AND PRACTICE Table 19.17 Van Ijzeren’s decomposition of the Fisher price index Period t P F 0.4011 0.0589 0.0376 0.0365 v F 1* Dp1 0.0178 0.0117 0.0130 0.0110 v F 2* Dp2 0.1882 0.1302 0.0342 0.0310 v F 3* Dp3 0.0579 0.0243 0.0110 0.0000 v F 4* Dp4 0.0341 0.0274 0.0521 0.0904 v F 5* Dp5 0.1822 0.0952 0.0629 0.0311 v F 6* Dp6 0.0109 0.0091 0.0123 0.0191 Table 19.18 The Lowe and Young indices, the ﬁxed base Laspeyres, Paasche and Fisher indices, and the chained Laspeyres, Paasche and Fisher indices Period t 3 4 5 P Lo 1.0000 1.0074 1.0706 P Y 1.0000 0.9396 0.9794 P L 1.0000 0.9784 1.0105 P P 1.0000 0.9466 0.8457 P F 1.0000 0.9624 0.9244 P LCH 1.0000 0.9784 0.9751 P PCH 1.0000 0.9466 0.8818 P FCH 1.0000 0.9624 0.9273 i 19.35 The chain links for the Fisher price index will be decomposed into price change components using the formulae (19.4) to (19.6), listed above.
The results of the decomposition are listed in Table 19.17.

Thus PF 1 is the percentage change in the Fisher ideal chain link going from period t 1 to t and the Van Ijzeren decomposition factor vFi* Dpi is the contribution to the total percentage change of the change in the ith price from pt1 to for i=1, 2, .

.

.

, 6.
.

.

, 6.

19.36 Comparing the entries in Tables 19.16 and 19.17, it can be seen that the diﬀerences between the Diewert and Van Ijzeren decompositions of the Fisher price index are very small.

The maximum absolute dif- ference between the vFiDpi and vFi* Dpi is only 0.0018 (about 0.2 percentage points) and the average absolute diﬀerence is 0.0003.
This is somewhat surprising given the very diﬀerent nature of the two decompositions.24 As was mentioned in footnote 43 of Chapter 16, the Van Ijzeren decomposition of the chain Fisher quantity index is used by the Bureau of Economic Analysis in the United States.25 The Lowe and Young indices 19.37 Recall that the Lowe index was deﬁned by equation (15.15) in Chapter 15.
If it is desired to com- pare the prices in period t with those in period 0, the formula for the Lowe index is given by equation (19.7) below: PLo( p1, pt, qb) P 6 i=1 iqb pt i 6 P i=1 i qb p0 i t=1, 2, .

.

.

, 5 (19:7) 1, qb 2, .

.

.

, qb 6] where qb [qb is the quantity vector per- taining to a base period b which is prior to period 0, the base period for prices.
This index will be computed for periods t equal to 3 to 5 for the artiﬁcial data set where the quantity reference period b is taken to be 24 Reinsdorf, Diewert and Ehemann (2002) nevertheless show that the terms in the two decompositions approximate each other to the second order around any point where the two price vectors are equal and where the two quantity vectors are equal.

25 See Moulton and Seskin (1999), and Ehemann, Katz and Moulton (2002).
352 period 1 and the price reference period 0 is taken to be period 3; see the column with the heading PLo in Table 19.18.

19.38 For comparison purposes, the ﬁxed base Laspeyres, Paasche and Fisher indices are also calcu- lated for periods 3, 4 and 5, where period 3 is treated as the base period; see the columns with the headings PL, PP and PF respectively.
The chained Laspeyres, Paasche and Fisher indices are also calculated for periods 3, 4 and 5, and listed in Table 19.18; see the columns with the headings PLCH, PPCH and PFCH respectively.

Table 19.18 shows that the Lowe index is higher than all six of these comparison indices in periods 4 and 5.
In particular, the Lowe index PLo is greater than the ﬁxed base Laspeyres index PL for periods 4 and 5, which is consistent with the inequality (15.37) in Chapter 15, where it was argued that the Lowe index would exceed the Laspeyres index if there were long-run trends in prices.

Compared with the preferred ﬁxed base or chained Fisher ideal target indices, PF or PFCH, the Lowe index has a considerable upward bias for this trending artiﬁ- cial data set.
19.39 The Young index was deﬁned by equation (15.48) in Chapter 15 and, for convenience, this deﬁni- tion is repeated below: PY ( p0, pt, sb) P n i=1 i ( pt sb i=p0 i ) (19:8) The base period b expenditure shares for the commodities are the sb i in equation (19.8) and the price reference period is period 0.
This Young index will be computed for periods t equal to 3 to 5 for the artiﬁcial data set, where the quantity reference period b is taken to be period 1 and the price reference period 0 is taken to be period 3; see the column with the heading PY in Table 19.18.

19.40 For periods 4 and 5, the Young index is below the corresponding values for the ﬁxed base Laspeyres index.26 For period 4, the Young index is 0.9396, which is below the corresponding value for the Fisher index, which is 0.9624.
However, for period 5, the Young index is 0.9794, which is above the corresponding values for 26 It is noted in Chapter 15 that the Young index can be above or below the corresponding ﬁxed base Laspeyres index, depending on the responsiveness of expenditure shares to changes in prices.

the two target Fisher indices, which are 0.9244 for the ﬁxed base index and 0.9273 for the chained index.
Thus, although the direction of the bias in the Young index is not always the same, it can be seen that it has substantial biases for the artiﬁcial data set compared to the pre- ferred target indices.

Mid-year indices based on the Lowe formula 19.41 Recall the Lowe index formula (19.7).

In most applications of the formula by statistical agencies, the quantity vector q will be taken from a period that is prior to the base period for prices, which is period 1 in the artiﬁcial data set.
It is also possible, however, to use the formula as a type of mid-year index, where the reference quantity vector q could be taken to be an average of the quantity vectors pertaining to periods 1 to 5.

This pos- sible use of the formula will be explored in the present section.

Thus the ﬁrst Lowe index, PLo1, sets q in formula (19.7) equal to q1, the period 1 quantity vector in the artiﬁcial data set.
This index turns out to be identical to the ﬁxed base Laspeyres index PL, which was listed earlier in Table 19.4.

The second Lowe index, PLo2, sets q in formula (19.7) equal to the average of the period 1 and 2 quantity vectors, (1=2)(q1+q2).27 The third Lowe index, PLo3, sets q equal to the average of the period 1 to 3 quantity vectors, (1=3)(q1+q2+q3).

The fourth Lowe index, PLo4, sets q equal to the average of the period 1 to 4 quantity vectors, (1=4)(q1+q2+q3+q4).
Finally, the ﬁfth Lowe index, PLo5, sets q equal to the average of the period 1 to 5 quantity vectors, (1=5)(q1+q2+q3+ q4+q5).28 The resulting ﬁve Lowe type indices are listed in Table 19.19.
19.42 The mid-year index PMY PLo( p1, pt, q3) was deﬁned in paragraphs 15.49 to 15.53 of Chapter 15; it is a Lowe type index with the ‘‘representative’’ quantity vector q chosen to be q3, the quantity vector that per- tains to the middle period in the span of periods under consideration (which is periods 1 to 5 in the numerical example).
It is listed as the seventh column in Table 19.18.29 The mid-year index and the ﬁve Lowe indices are compared to two of the ‘‘best’’ target indices, the chain To¨ rnqvist and chain Fisher indices, PT and PF listed in the last two columns of Table 19.19.
19.43 From Table 19.19, it can be seen that none of the Lowe type indices (or the mid-year index) are very close to the two target indices (the chain To¨ rnqvist and chain Fisher) for all periods.30 With less extreme data sets, however, it is quite possible that the ﬁfth Lowe 27 This is the Lowe index for the artiﬁcial data set, which will probably be the most comparable to the type of Lowe index currently computed by statistical agencies.
28 This is Walsh’s (1901, p. 431) multi-year ﬁxed basket index, where the quantity vector is chosen to be the arithmetic average of the quantity vectors in the time period under consideration.

29 It can be veriﬁed that if there are exact linear time trends in the quantity data, then the mid-year index PMY will be exactly equal to the ﬁfth Lowe index, PLo5.

30 The fourth Lowe index PLo4 and the mid-year index PMY appear to be the closest to the target indices.
PRICE INDICES USING AN ARTIFICIAL DATA SET index and the mid-year index could form adequate approximations to the target indices.

19.44 With strong trends in the price data and normal consumer substitution responses, it is unlikely that Lowe type indices, based on averages of the quantity data pertaining to the ﬁrst few periods in a long time series of data, will be able to provide an adequate approximation to a chained superlative index.
In gen- eral, this type of Lowe index will suﬀer from an upward bias compared to the target index, as can be seen from Table 19.19.

Young-type indices 19.45 Recall that the Young index was deﬁned by equation (15.48) in Chapter 15, or equation (19.8) above.

If it is desired to compare the prices in period t with those in period 1, the formula for the Young index is given by equation (19.9): PY ( p1, pt, sb) P 6 i=1 i ( pt sb i=p1 i ) for t=1, 2, .

.

.

, 5 (19:9) 1, .

.

.


sb 6] where the expenditure share vector sb [sb is ‘‘representative’’ for the span of periods under con- sideration.

In most applications of the formula by sta- tistical agencies, the base period expenditure share vec- tor sb will be taken from a period that is prior to the base period for prices, which is period 1 in the artiﬁcial data set.
For illustrative purposes, rather than adding new data to the artiﬁcial data set, the reference share vector sb will be taken to be an average of the expenditure share vectors pertaining to periods 1 to 5.

Thus, the ﬁrst Young-type index, PY1, sets sb in formula (19.9) equal to s1, the period 1 expenditure share vector in the artiﬁcial data set.

This index turns out to be identical to the ﬁxed base Laspeyres index PL, shown in Table 19.4.
The second Young-type index, PY2, sets sb in formula (19.9) equal to the average of the period 1 and 2 share vectors, (1=2)(s1+s2).

The third Young-type index, PY3, sets sb equal to the average of the period 1 to 3 share vectors, (1=3)(s1+s2+s3).

The fourth Young-type index, PY4, sets sb equal to the average of the period 1 to 4 share vectors, (1=4)(s1+s2+s3+s4).

Finally, the ﬁfth Young- type index, PY5, sets sb equal to the average of the period 1 to 5 share vectors, (1=5)(s1+s2+s3+s4+s5).
The resulting ﬁve Young type indices are listed in Table 19.20 below.

These indices are compared to two of the ‘‘best’’ target indices, the chained To¨ rnqvist and chained Fisher indices, PT and PF listed in the last two columns of Table 19.20.

19.46 Table 19.20 shows that all the Young-type indices exhibit a substantial upward bias compared to the target chain To¨ rnqvist and Fisher indices, PT and PF.
Comparing Table 19.19 with Table 19.20, it can be seen that the bias in the Young-type indices becomes bigger as the expenditure shares become more repre- sentative of all ﬁve periods, whereas the upward bias in the Lowe-type indices tends to become smaller as the reference quantity vector became more representative of all periods.
353 CONSUMER PRICE INDEX MANUAL: THEORY AND PRACTICE Table 19.19 The ﬁve Lowe indices, the mid-year index, and the To¨ rnqvist and Fisher chain indices Period t P Lo1 1.0000 1.4200 1.3450 1.3550 1.4400 P Lo2 1.0000 1.4010 1.3366 1.3485 1.4252 P Lo3 1.0000 1.3641 1.2851 1.2824 1.3444 P Lo4 1.0000 1.3068 1.2142 1.1926 1.2321 P Lo5 1.0000 1.2267 1.1234 1.0801 1.0868 P MY 1.0000 1.3055 1.2031 1.1772 1.2157 P T 1.0000 1.4052 1.3112 1.2624 1.2224 Table 19.20 The ﬁve Young-type indices and the To¨ rnqvist and Fisher chain indices Period t P Y 1 1.0000 1.4200 1.3450 1.3550 1.4400 P Y 2 1.0000 1.5148 1.3567 1.3526 1.4632 P Y 3 1.0000 1.4755 1.3765 1.3917 1.4918 P Y 4 1.0000 1.4409 1.3943 1.4267 1.5173 P Y 5 1.0000 1.4355 1.4144 1.4584 1.5482 P T 1.0000 1.4052 1.3112 1.2624 1.2224 P F 1.0000 1.4011 1.3185 1.2689 1.2226 P F 1.0000 1.4011 1.3185 1.2689 1.2226 19.47 Note that the Young type indices PY2 to PY5 are all bigger in magnitude than PY1, which is the ordinary ﬁxed base Laspeyres index.
It must be recog- nized, however, that these Young type indices are not the type of Young index that is computed by statistical agencies, in which the weight reference period precedes the price reference period.

As discussed in paragraphs 19.39 to 19.42, this latter type of Young index could be above or below the corresponding ﬁxed base Laspeyres index.
19.48 The results of this section and the previous one can be summarized as follows: it appears to be a useful exercise to attempt to ﬁnd quantity weights for the Lowe formula that are representative for the entire period covered by the index, but it does not appear to be useful to do the same for the Young formula.

354 ELEMENTARY INDICES Introduction 20.1 In all countries, the calculation of a consumer price index (CPI) proceeds in two (or more) stages.
In the ﬁrst stage of calculation, elementary price indices are estimated for the elementary expenditure aggregates of a CPI.

In the second and higher stages of aggregation, these elementary price indices are combined to obtain higher-level indices using information on the expendi- tures on each of the elementary aggregates as weights.

An elementary aggregate consists of the expenditures on a small and relatively homogeneous set of products deﬁned within the consumption classiﬁcation used in the CPI.
Samples of prices are collected within each elementary aggregate, so that elementary aggregates serve as strata for sampling purposes.

20.2 Data on the expenditures, or quantities, of the diﬀerent goods and services are typically not available within an elementary aggregate.

As there are no quan- tity or expenditure weights, most of the index number theory outlined in Chapters 15 to 19 is not directly applicable.
As was noted in Chapter 1, an elementary price index is a more primitive concept that relies on price data only.

20.3 The question of what is the most appropriate formula to use to estimate an elementary price index is considered in this chapter.

The quality of a CPI depends heavily on the quality of the elementary indices, which are the basic building blocks from which CPIs are constructed.
20.4 As is explained in Chapter 6, compilers have to select representative products within an elementary aggregate and then collect a sample of prices for each of the representative products, usually from a sample of diﬀerent outlets.

The individual products for which prices are actually collected are described as the sampled products.

Their prices are collected over a succession of time periods.

An elementary price index is therefore typically calculated from two sets of matched price observations.
In most of this chapter,1 it is assumed that there are no missing observations and no changes in the quality of the products sampled so that the two sets of prices are perfectly matched.

The treatment of new and disappearing products, and of quality change, is a separate and complex issue that is discussed in detail in Chapters 7, 8 and 21 of this manual.
20.5 Even though quantity or expenditure weights are usually not available to weight the individual ele- 1 The problem of sample attrition and the lack of matching over time is discussed brieﬂy in the context of classiﬁcation issues in paragraphs 20.23 to 20.37.

20 mentary price quotes, it is useful to consider an ideal framework where expenditure information is available.

This is done in the next section.


The problems involved in aggregating narrowly deﬁned price quotes over time are also discussed in that section.

Thus the discussion provides a theoretical target for ‘‘practical’’ elementary price indices that are constructed using only information on prices.

20.6 Paragraphs 20.23 to 20.37 provide some dis- cussion about the diﬃculties involved in picking a suitable level of disaggregation for the elementary aggre- gates.
Should the elementary aggregates have a regional dimension in addition to a product dimension?

Should prices be collected from retail outlets or from house- holds?

These are the types of question discussed in this section.

20.7 Paragraphs 20.38 to 20.45 introduce the main elementary index formulae that are used in practice, and paragraphs 20.46 to 20.57 develop some numerical relationships between the various indices.
20.8 Chapters 15 to 17 develop the various ap- proaches to index number theory when information on both prices and quantities is available.

It is also possible to develop axiomatic, economic or sampling (stochastic) approaches to elementary indices, and these three approaches are discussed below in para- graphs 20.58 to 20.70, 20.71 to 20.86, and 20.87, respec- tively.
20.9 Paragraphs 20.88 to 20.99 look at some of the recent scanner data literature that computes elementary aggregates using both price and quantity information.

20.10 Paragraphs 20.100 to 20.111 develop a simple statistical approach to elementary indices that resembles a highly simpliﬁed hedonic regression model.
The con- cluding section presents an overview of the various results.2 Ideal elementary indices 20.11 The aggregates covered by a CPI or a pro- ducer price index (PPI) are usually arranged in the form of a tree-like hierarchy, such as the Classiﬁcation of Individual Consumption according to Purpose (COI- COP)3 or the Nomenclature ge´ ne´ rale des Activite´ s 2 This chapter draws heavily on the recent contributions of Dale´ n (1992), Balk (1994; 1998b; 2002) and Diewert (1995a; 2002c).
3 Triplett (2003, p. 160) is quite critical of the COICOP classiﬁcation scheme and argues that economic theory and empirical analysis should be used to derive a more appropriate CPI classiﬁcation scheme.

It is nevertheless very diﬃcult to coordinate a classiﬁcation scheme that can be used by all countries.

355 CONSUMER PRICE INDEX MANUAL: THEORY AND PRACTICE e´ conomiques dans les Communaute´ s Europe´ ennes [General Industrial Classiﬁcation of Economic Activities within the European Communities] (NACE).
Any aggre- gate is a set of economic transactions pertaining to a set of commodities over a speciﬁed time period.

Every economic transaction relates to the change of ownership of a speciﬁc, well-deﬁned commodity (good or service) at a particular place and date, and comes with a quantity and a price.

The price index for an aggregate is calcu- lated as a weighted average of the price indices for the sub-aggregates, the (expenditure or sales) weights and type of average being determined by the index formula.
One can descend in such a hierarchy as far as available information allows the weights to be decomposed.

The lowest-level aggregates are called elementary aggregates.
They are basically of two types: those for which all detailed price and quantity infor- mation is available; those for which the statistician, considering the operational cost or the response burden of getting detailed price and quantity information about all the transactions, decides to make use of a representative sample of commodities or respondents.

20.12 The practical relevance of studying this topic is large.
Since the elementary aggregates form the building blocks of a CPI or a PPI, the choice of an inappropriate formula at this level can have a tremendous impact on the overall index.

20.13 In this section, it will be assumed that detailed price and quantity information for all transactions per- taining to the elementary aggregate for the two time periods under consideration is available.

This assump- tion allows us to deﬁne an ideal elementary aggregate.
Subsequent sections will relax this strong assumption about the availability of detailed price and quantity data on transactions, but it is necessary to have a theoreti- cally ideal target for the ‘‘practical’’ elementary index.

20.14 The detailed price and quantity data, although perhaps not available to the statistician, are in principle available in the outside world.
It is frequently the case that at the respondent level (i.e., at the outlet or ﬁrm level) some aggregation of the individual transactions information has been executed, usually in a form that suits the respondent’s ﬁnancial or management infor- mation system.

This level of information that is deter- mined by the respondent could be called the basic information level.

It is, however, not necessarily the ﬁnest level of information that could be made available to the price statistician.
One could always ask the respon- dent to provide more disaggregated information.

For instance, instead of monthly data one could ask for weekly data; or, whenever appropriate, one could ask for regional instead of global data; or one could ask for data according to a ﬁner commodity classiﬁcation.
The only natural barrier to further disaggregation is the individual transaction level.4 20.15 It is now necessary to discuss a problem that arises when detailed data on individual transactions are available, either at the level of the individual household 4 See Balk (1994).

356 i and qt or at the level of an individual outlet.

Recall that Chapter 15 introduces the price and quantity indices, P( p0, p1, q0, q1) and Q( p0, p1, q0, q1).
These (bilateral) price and quantity indices decompose the value ratio V1/V 0 into a price change part P( p0, p1, q0, q1) and a quantity change part Q( p0, p1, q0, q1).

In this framework, it is taken for granted that the period t price and quantity for commodity i; pt i respectively, are well deﬁned.

These deﬁnitions are not, however, straightforward since individual consumers may purchase the same item during period t at diﬀerent prices.
Similarly, if one considers the sales of a particular shop or outlet that sells to con- sumers, the same item may sell at very diﬀerent prices during the course of the period.
Hence before a tradi- tional bilateral price index of the form P( p0, p1, q0, q1) considered in previous chapters of this manual can be applied, a non-trivial time aggregation problem must be resolved in order to obtain the basic prices pt i and quantities qt i that are the components of the price vectors p0 and p1 and the quantity vectors q0 and q1.
20.16 Walsh5 and Davies (1924; 1932) suggested a solution to this time aggregation problem: in their view, the appropriate quantity at this very ﬁrst stage of aggregation is the total quantity purchased of the nar- rowly deﬁned item and the corresponding price is the value of purchases of this item divided by the total amount purchased, which is a narrowly deﬁned unit value.
In more recent times, other researchers have adop- ted the Walsh and Davies solution to the time aggre- gation problem.6 Note that the following advantages: The quantity aggregate is intuitively plausible, being the total quantity of the narrowly deﬁned item pur- chased by the household (or sold by the outlet) during the time period under consideration.
solution has this The product of the price times quantity equals the total value purchased by the household (or sold by the outlet) during the time period under consideration.

20.17 The above solution to the time aggregation problem will be adopted as the concept for the price and quantity at this very ﬁrst stage of aggregation.

This leaves open the question of how long the time period should be over which the unit value is calculated.

This question will be considered in the following section.
20.18 Having decided on an appropriate theoretical deﬁnition of price and quantity for an item at the very 5 Walsh explained his reasoning as follows: Of all the prices reported of the same kind of article, the average to be drawn is the arithmetic; and the prices should be weighted according to the relative mass quantities that were sold at them (Walsh (1901, p. 96)).
Some nice questions arise as to whether only what is consumed in the country, or only what is produced in it, or both together are to be counted; and also there are diﬃculties as to the single price quotation that is to be given at each period to each commodity, since this, too, must be an average.

Throughout the country during the period a commodity is not sold at one price, nor even at one wholesale price in its principal market.
Various quantities of it are sold at diﬀerent prices, and the full value is obtained by adding all the sums spent (at the same stage in its advance towards the consumer), and the average price is found by dividing the total sum (or the full value) by the total quantities (Walsh (1921a, p. 88)).

6 See for example Szulc (1987, p. 13), Dale´ n (1992, p. 135), Reinsdorf (1994), Diewert (1995a, pp.

20–21), Reinsdorf and Moulton (1997), Balk (2002) and Richardson (2003).
1, pt lowest level of aggregation (i.e., a narrowly deﬁned unit value and the total quantity sold of that item at the individual outlet, or the total quantity purchased by a single household or a group of households), it is neces- sary to consider how to aggregate these narrowly deﬁned elementary prices and quantities into an overall elementary aggregate.

Suppose that there are M lowest- level items or speciﬁc commodities in this chosen ele- mentary category.
Denote the period t quantity of item m by qt m and the corresponding time-aggregated unit value by pt m for t=0,1 and for items m=1, 2, .

.

.

, M. Deﬁne the period t quantity and price vectors as qt [qt 1, qt 2, .

.

.

, qt 2, .

.

.

, pt M] and pt [ pt for t=0,1.


It is now M] necessary to choose a theoretically ideal index number formula P( p0, p1, q0, q1) that will aggregate the individual item prices into an overall aggregate price relative for the M items in the chosen elementary aggregate.

This pro- blem of choosing a functional form for P( p0, p1, q0, q1) is identical to the overall index number problem that is addressed in Chapters 15–17.
In these previous chapters, four diﬀerent approaches to index number theory are studied, and speciﬁc index number formulae are seen as being ‘‘best’’ from each perspective.

From the viewpoint of ﬁxed basket approaches, the Fisher (1922) and Walsh (1901) price indices, PF and PW, appear to be ‘‘best’’.

From the viewpoint of the test approach, the Fisher index appears to be ‘‘best’’.
From the viewpoint of the sto- chastic approach to index number theory, the To¨ rnqvist– Theil (1967) index number formula PT emerges as being ‘‘best’’.

Finally, from the viewpoint of the economic approach to index number theory, the Walsh price index PW, the Fisher ideal index PF and the To¨ rnqvist–Theil index number formula PT are all regarded as being equally desirable.
It is also shown that these three index number formulae numerically approximate each other very closely, and so it does not matter very much which of these alternative indices is chosen.7 Hence, the theo- retically ideal elementary index number formula is taken the three formulae PF ( p0, p1, q0, q1), to be one of PW ( p0, p1, q0, q1) or PT ( p0, p1, q0, q1) where the period t quantity of item m, qt m, is the total quantity of that nar- rowly deﬁned item purchased by the household during period t (or sold by the outlet during period t) and the corresponding price for item m is pt m, the time-aggregated unit value, for t=0, 1 and for items m=1, 2, .
.

.

, M.8 20.19 Various ‘‘practical’’ elementary price indices are deﬁned in paragraphs 20.38 to 20.45.

These indices 7 Theorem 5 in Diewert (1978, p. 888) shows that PF, PT and PW approximate each other to the second order around an equal price and quantity point; see Diewert (1978, p. 894), Hill (2002) and Chapter 19 for some empirical results.
8 Of course, all these ideal elementary index number formulae require current period quantity (or expenditure) weights and thus are not usually ‘‘practical’’ formulae that can be used to produce the usual type of month-to-month CPI.

Nevertheless, as statistical agencies introduce superlative indices on a retrospective basis, it may be pos- sible to obtain more current information on weights, at least at higher levels of aggregation; see Greenlees (2003).
Gudnason (2003, p. 16) also gives some examples where the Icelandic CPI obtains enough information to be able to calculate some elementary indices using a superlative formula.

In any case, a target index is required at just as one is required at higher levels of the elementary level aggregation.
ELEMENTARY INDICES do not have quantity weights and thus are functions only of the price vectors p0 and p1, which contain time- aggregated unit values for the M items in the elementary aggregate for periods 0 and 1.
Thus when a practical elementary index number formula, say PE( p0, p1), is compared to an ideal elementary price index, say the Fisher price index PF ( p0, p1, q0, q1), then obviously PE will diﬀer from PF because the prices are not weighted according to their economic importance in the prac- tical elementary formula.9 Call this diﬀerence between the two index number formulae formula approximation error.
20.20 Practical elementary indices are subject to other types of error as well: The statistical agency may not be able to collect information on all M prices in the elementary aggre- gate; i.e., only a sample of the M prices may be col- lected.

Call the resulting divergence between the incomplete elementary aggregate and the theoretically ideal elementary index the sampling error.