metadata: authors: 0: author: Andree,Bo Pieter Johannes
1: author: Pape,Utz Johann
count: Papua New Guinea,World
docna: 0: docna: Machine Learning Imputation of High
            Frequency Price Surveys in Papua New Guinea
docty: Policy Research Working Paper
abstracts: cdata!: Capabilities to track fast-moving
            economic developments re-main limited in many regions of the
            developing world. This complicates prioritizing policies
            aimed at supporting vulnerable populations. To gain insight
            into the evolution of fluid events in a data scarce context,
            this paper explores the ability of recent machine-learning
            advances to produce continuous data in near-real-time by
            imputing multiple entries in ongoing surveys. The paper
            attempts to track inflation in fresh produce prices at the
            local market level in Papua New Guinea, relying only on
            incomplete and intermittent survey data. This application is
            made challenging by high intra-month price volatility, low
            cross-market price correlations, and weak price trends. The
            modeling approach uses chained equations to produce an
            ensemble prediction for multiple price quotes
            simultaneously. The paper runs cross-validation of the
            prediction strategy under different designs in terms of
            markets, foods, and time periods covered. The results show
            that when the survey is well-designed, imputations can
            achieve accuracy that is attractive when compared to
            costly&#x2013;and logistically often
            infeasible&#x2013;direct measurement. The methods have
            wider applicability and could help to fill crucial data gaps
            in data scarce regions such as the Pacific Islands,
            especially in conjunction with specifically designed
            continuous surveys.
display_title: Machine Learning Imputation of High
            Frequency Price Surveys in Papua New Guinea
listing_relative_url: /research/2023/09/34156147/machine-learning-imputation-high-frequency-price-surveys-papua-new-guinea
topicv3: Agriculture and Food Security m1326205 435,Food Price Analysis m1326440 200,Economic Shocks and Vulnerability m1326506 702,Inflation m1326709 0870F4727ACED1CF8525815C00471347,Macroeconomic Monitoring m1326902 440
publishtoextweb_dt: 2023-09-05T00:00:00Z
dois: 10.1596/1813-9450-10558
origu: Off of Sr VP Dev Econ/Chief Econ (DECVP)
author_keys: 0: author_key: 1539948
1: author_key: 1154467
model_extractions: Policy Research Working Paper 10559 Machine Learning Imputation of High Frequency Price Surveys in Papua New Guinea _Bo Pieter Johannes Andrée Utz Johann Pape_ Development Data Group Agriculture and Food Global Practice & Poverty and Equity Global Practice September 2023
Policy Research Working Paper 10559

# **Abstract**

Capabilities to track fast-moving economic developments re-main limited in many regions of the developing world. This complicates prioritizing policies aimed at supporting vulnerable populations. To gain insight into the evolution of fluid events in a data scarce context, this paper explores the ability of recent machine-learning advances to produce continuous data in near-real-time by imputing multiple entries in ongoing surveys. The paper attempts to track inflation in fresh produce prices at the local market level in Papua New Guinea, relying only on incomplete and intermittent survey data. This application is made challenging by high intra-month price volatility, low cross-market price correlations, and weak price trends. The modeling approach uses chained equations to produce an ensemble prediction for multiple price quotes simultaneously. The paper runs cross-validation of the prediction strategy under different designs in terms of markets, foods, and time periods covered. The results show that when the survey is well-designed, imputations can achieve accuracy that is attractive when compared to costly–and logistically often infeasible–direct measurement. The methods have wider applicability and could help to fill crucial data gaps in data scarce regions such as the Pacific Islands, especially in conjunction with specifically designed continuous surveys.

This paper is a product of the Development Data Group, Development Economics, the Agriculture and Food Global Practice; and the Poverty and Equity Global Practice. It is part of a larger effort by the World Bank to provide open access to its research and make a contribution to development policy discussions around the world. Policy Research Working Papers are also posted on the Web at http://www.worldbank.org/prwp. The authors may be contacted at bandree@worldbank.org and upape@worldbank.org.

_The Policy Research Working Paper Series disseminates the findings of work in progress to encourage the exchange of ideas about development issues. An objective of the series is to get the findings out quickly, even if the presentations are less than fully polished. The papers carry the names of the authors and should be cited accordingly. The findings, interpretations, and conclusions expressed in this paper are entirely those of the authors. They do not necessarily represent the views of the International Bank for Reconstruction and Development/World Bank and its affiliated organizations, or those of the Executive Directors of the World Bank or the governments they represent._ Produced by the Research Support Team
# **Machine Learning Imputation of High Frequency Price Surveys in Papua New Guinea**

_By_ Bo Pieter Johannes Andr ́ee and Utz Johann Pape<sup>∗</sup> JEL: C01, C14, C25, C53, O10.

Keywords: Inflation, Agriculture and Food Security, Food Price Analysis, Economic Shocks and Vulnerability, Macroeconomic Monitoring.

> ∗ Bo Pieter Johannes Andr ́ee, The World Bank, Development Economics, Data Group, can be contacted at bandree(at)worldbank.org. Utz Johann Pape, The World Bank, Poverty & Equity Global Practice, East Asia and Pacific, as well as University of G ̈ottingen, can be contacted at upape(at)worldbank.org. The findings, interpretations, and conclusions expressed in this paper are entirely those of the authors. They do not necessarily represent the views of the World Bank and its affiliated organizations, or those of the Executive Directors of the World Bank or the governments they represent.

_1_
_ANDR EE_<sup>_ ́_</sup> _AND PAPE_ _SEPTEMBER 5, 2023_ _2_

# **I. Introduction**

Statistical agencies around the world are increasingly interested in the use of machine learning in the production of official statistics. In particular, the area of missing data imputation is one of potentially promising applications. A recent survey on the use of machine learning methods in official statistics commissioned by the United Nations Economic Commission and conducted at selected national and international statistical institutions revealed that missing data imputation was second in a ranking of promising areas (Beck et al., 2022). Real-time imputation of economic data may also hold the key to enabling reliable forecasting and monitoring of risks in humanitarian settings where primary data often cannot be collected (Andr ́ee et al., 2020; Wang et al., 2020, 2022), or in development contexts where large scale data operations are typically carried out on an infrequent basis (Mahler et al., 2021).

Traditionally, surveys have been deployed as self-contained data gathering operations aimed at capturing a snapshot of an evolving population statistic such as the poverty rate, market sentiment, or the consumer price index. Developing such one-time analyses has been the bread and butter task of economists for decades, and the go-to approach for policy makers to inform their next actions. The issue of missing data has traditionally been approached from the angle of correcting for the bias and uncertainty that arise in this analytical context. In particular, the work of Rubin (1976); Campion and Rubin (1989); Rubin (1996); Little and Rubin (2012); van Buuren (2012) on multiple imputation has provided important answers to the question of how to deal with non-response when estimating economic relationships.

Increasingly, however, economists and policy makers are looking for continuous insight, as shown by the surge in literature on “now-casting” and real-time indicators (Khan et al., 2022). The literature has put forward many promising applications, but now-casting composite variables is specifically hard as it involves tracking the evolution of multiple contributing factors in a structured manner.

As an example, economists looking at inflation generally track a price index, comprising the combined prices of a consistent basket of important goods. Nonresponses in the price data gathered for the entire basket are highly problematic. Inflation calculation requires all prices in the basket to be observed without bias. Thus, an accurate assumption for the value of the non-response is necessary. For this reason, price surveys traditionally follow a deliberate sampling and measurement process (Reinsdorf et al., 2009) that minimizes measurement errors, missing price quotes, or biases that stem from the locations or methods of measurement, as all are sources of error (Baker, 1996; Lebow and Rudd, 2003; Greenlees and McClelland, 2010). The deliberate approach makes traditional price data gathering methods robust, but highly inflexible, and typically not suitable for tracking inflation in near-real-time or high-frequency settings, except in few countries with exceptional statistical capacity.

Short-burst rapid surveys are increasingly relied on to complement traditional
_MACHINE LEARNING AND HIGH-FREQUENCY SURVEYS IN PAPUA NEW GUINEA 3_ survey data, for instance by leveraging high-frequency phone interviews to collect data to inform developmental responses to emergencies (Hoogeveen and Pape, 2019). In practice, a rapid survey system hardly produces complete data, particularly when deployed in difficult settings such as during economic turmoil, conflict, or natural disasters, when it is particularly important to understand possible drastic shifts in economic variables. A great deal of innovation is aimed at the question of how to design such surveys to ensure sufficient response rates, and how to process the data to produce correct estimates (Pape and Wollburg, 2019; Pape, 2021; Khamis et al., 2021).

To overcome some of the challenges with high-frequency surveying, Andr ́ee (2021) developed an approach for real-time imputation of ongoing surveys. Specifically, the paper proposed a matrix-completion algorithm based on multiple machine learning models that simultaneously estimates missing entries using information contained in other responses. This works well when the survey tracks multiple correlated variables, and is specifically suitable to impute a price index, as prices of different goods are typically interrelated. The imputation can be applied in high-frequency settings in which only incomplete and intermittent data can be collected.

The aim of this paper is to continue the line of investigation into the ability of machine-learning techniques to impute ongoing surveys in near-real-time and produce continuous data that yield insights into the evolution of possibly fluid events in data-scarce contexts. The paper focuses on a challenging nowcasting objective. It attempts to track inflation in fresh produce prices at the local market level in Papua New Guinea (PNG) using monthly survey data obtained from the International Food Policy Research Institute (IFPRI). The application is made particularly challenging by high intra-month price volatility in fresh produce items, low cross-market price correlations owing to a lack of overall market integration, and weak price trends. The application cross-validates the imputation strategy under different designs in terms of numbers of markets, food items and time periods covered, and shows that when the survey is well-designed, imputations can achieve accuracy that is attractive when compared to costly– and logistically often infeasible–direct measurement. The localized statistics are shown to provide a new granular view on recent food price inflation dynamics in PNG leading up to and after the outbreak of the pandemic, and more recently the conflict in Ukraine.

The application builds on the original algorithm described by Andr ́ee (2021) but suggests methodological improvements that produce faster and more accurate results, particularly in lower data availability settings. The reduced computing time enables the paper to process a higher number of food items. The resulting estimates cover up to 27 fresh food items, across 8 markets, for the period from mid-2009 to July 2022. This is up from an average of 7 and maximum of 16 price items processed by Andr ́ee (2021). The paper also shows that the estimation methods can be applied to different price surveys, including those from country
_ANDR EE_<sup>_ ́_</sup> _AND PAPE_ _SEPTEMBER 5, 2023_ _4_ systems. To handle the relatively large temporal gaps in the IFPRI data, the paper incorporates exchange rate data, showing that the methods can also produce estimates of unofficial parallel-market exchange rates that allow dollarizing the price data streams in real-time. Finally, the paper shows that the methods are applied successfully to short time series, opening the door to piloting the methods in conjunction with ongoing mobile phone surveys in a high-frequency setting.

The application to PNG data is valuable as formal traditional methods suitable for high frequency price tracking are not implemented in the Pacific Island region for a number of reasons, including low capacity, challenging geography and incomplete digitalization of market price information. Furthermore, prices collected with traditional methods are released often only after the data are already outdated. The Pacific Islands are generally import dependent for food products such as grains, meats, dairy products and vegetable oils, which all rose sharply during the previous major global food price spike in 2008 (McGregor et al., 2009). However, without adequate price monitoring capabilities, it is difficult to assess how the development context is changing in the region while countries across the world are grappling with falling living standards (Egger et al., 2021), high inflation (Etang et al., 2022; World Bank, 2022), and volatility in commodity prices (World Bank Group, 2022). The paper concludes that the explored methods may have wider applicability and could help to fill crucial data gaps in the Pacific Islands, especially in conjunction with specifically designed continuous surveys.

The remainder of the paper is as follows. Section II discusses the survey data and imputation methodology. Section III presents imputation results for different setups in terms of the number of food items, markets and the temporal dimension of the survey data. Section IV concludes on the viability of the methods and the potential use of machine-learning augmented high frequency surveys in the Pacific Islands.

# **II. Methods**

# _A. Imputation strategy at a high level_

Table 1 visualizes the general missing data problem when gathering price data for the use of tracking inflation. In particular, the example considers tracking the basket-price in a simple three item setting, showing that even when there is a reasonable amount of price data, it may not be possible to observe the change in basket price at any given moment. The overall idea behind the suggested solution is to fill price gaps by leveraging prices of the same item in different markets or of other items in the same market.

Completing the missing entries is challenging using standard imputation tools. For instance, carrying the last observation forward suggests zero change and so results in a major bias in an application that aims to monitor inflation (price change), particularly in high inflationary environments. This is an issue when
_MACHINE LEARNING AND HIGH-FREQUENCY SURVEYS IN PAPUA NEW GUINEA 5_ Table 1—Example of the missing data problem.

_b_ 6 _c_ 6 _Note:_ Example of the missing data problem, three hypothetical vectors _A_ , _B_ , and _C_ that represent price series, with elements _at_ , _bt_ and _ct_ being individual price quotes indexed by time periods _t_ . Blank entries represent missing observations. The challenge is to estimate change rates ∆ _P_ of the basket price vector _P_ = _A_ + _B_ + _C_ that spans all _t_ = 1 _, . . . ,_ 6. Element _b_<sup>_∗_</sup> 4<sup>isanexampleoutlierpricewhichneedstobe</sup> removed and replaced with an estimate. _Source:_ Example has been taken from Andr ́ee (2021).

real-time estimates serve as proxies for economic indicators during fluid events when timely official data is not available and cannot be relied upon.

Univariate time series or multivariate regression techniques are also commonly deployed to address data gaps through prediction, but they do not efficiently exploit the information that is available. For instance, a univariate method applied to food item _A_ does not utilize the information available in items _B_ and _C_ . A regression model that explains _A_ based on ( _B, C_ ) is not possible, as there are 0 complete cases for the triplet, even though half of the observations are available in the example table.

Andr ́ee (2021) develops an imputation strategy suitable for the context. At a high level, the strategy starts off with the notion that the most valuable information for imputation is contained in the observed price ratios and should be utilized to fill in the blank spots. Specifically, taking the example of table 1, the problem could be stated as in which the goal is to estimate the L.H.S., using the information in price pairs on the R.H.S., so that ∆ _P_<sup>ˆ</sup> can be calculated. This information is utilized by using chained equations modeling (see van Buuren and Groothuis-Oudshoorn (2011); van Buuren (2012) for an in-depth treatment and implementation of the concept).

A simplified walk-through of the steps is as follows. The exact outline of the algorithm is provided in the appendix.

- 1) First, the missing price entries are filled based on prior assumptions. These prior assumptions could be based on expert opinion, be random (standard chained equation implementations typically start by filling entries using random draws of observations, see for instance the implementation of van Buuren and Groothuis-Oudshoorn (2011)); or be based on prior modeling (Andr ́ee (2021) used a combination of spatial and time series interpolation techniques, Andree (2022) uses a pattern-matching algorithm that fills gaps
_ANDR EE_<sup>_ ́_</sup> _AND PAPE_ _SEPTEMBER 5, 2023_ _6_ by replicating observed patterns).

- 2) Next, a regression model for column _A_ is estimated, using the other columns as predictors _A_ = _f_<sup>_A_</sup> ( _B, C_ ). Typically, the rows are selected so that the dependent variable consists fully of observations, while the covariates may be partially imputed data. The predictions of the model are used to replace the initial imputations in _A_ using predictions from this model.

- 3) Next, a model _B_ = _f_<sup>_B_</sup> ( _A, C_ ) is estimated and used to update imputations in _B_ in the same manner.

- 4) The process then cycles multiple times over all columns, and keeps updating the estimates for missing prices until the process converges.

It is noted by van Buuren and Groothuis-Oudshoorn (2011); van Buuren (2012) that convergence occurs relatively fast in this type of Markov Chain Monte Carlo (MCMC). A stopping criterion is provided by Andr ́ee (2021) based on convergence in cross-validation performance of the prediction models. Specifically, since updates to imputations always enter on the covariate side of the regressions, each regression benefits from improved data produced by the previous regression. Throughout the process of estimating, predicting, and then re-estimating, the regression models improve and can be cross-validated against holdout data at each step. When the cross-validation performance does not improve any further, the process can be terminated. Typically, there may be stochastic elements involved and the whole process is repeated several times to produce multiple imputations. Andr ́ee (2021) builds an ensemble predictor by aggregating the results.

# _B. Implementation_

There are two main ingredients needed to carry out the procedure. First, the sequential imputation process needs to be initialized at some, preferably reasonable, initial imputation. Second, the regression model used to make predictions throughout the process needs to be specified.

# Initialization

In real-time settings, the imputation process can be initialized with the estimates produced in the previous period, requiring only a simple one-step ahead extrapolation that can be generated using standard time series methods, see the Appendix for notes. This means that an initialization is only needed once, when the price imputation process is deployed. When the price imputation process is deployed in real-time, it can continue to run as a continuous MCMC process that integrates new observations on the fly and use them to update imputations on an ongoing basis.

Each imputation, _m_ , that underlies the final ensemble prediction, is initialized uniquely by adding a small disturbance term around these initial imputations to
_MACHINE LEARNING AND HIGH-FREQUENCY SURVEYS IN PAPUA NEW GUINEA 7_ reflect that initial imputations are uncertain. When the price imputation process is deployed in real-time, prior knowledge about heteroskedasticity can be obtained by modeling a Generalized Autoregressive Conditional Heteroskedastic (GARCH) error process. The initial disturbance term can then be drawn using observationlevel standard deviations. This means that when the price monitor is active in real-time, time-varying variance can be filtered and used to control the stochastic part of the initialization. See the Appendix for notes.

When a monitoring system is first deployed, and no previous results exist, a multi-step spatial time series interpolation approach can be taken to generate the initialization following the original implementation of Andr ́ee (2021).<sup>1</sup>

# Prediction Model

The regression models _f_<sup>_A_</sup> _, f_<sup>_B_</sup> _, ..._ involved are specified as follows (using the first as an example): The price data vectors are in long format, so that _A_ stacks the sub-vectors of market-level time series _A_ = _{a_ 1 _,_ 1 _, a_ 1 _,_ 2 _, ..., , a_ 1 _,T , ..., aJ,_ 1 _, aJ,_ 2 _, ..., , aJ,T }_ , with _aj,t_ being the price quote at location _j_ and time _t_ . Specifically, apart from the other prices items, a matrix of covariates is added. These include the spatial coordinates of markets, so that the model is able to interpolate spatially, as well as any prices vectors contained in ( _B, C_ ) that are at least 95% observed (minor gaps imputed using a structural time series model) so that the model has an accurate representation of observed temporal trends. Finally, the institutional exchange rate is added to inform the model about currency depreciation effects.<sup>2</sup>

> 1Recall that the data consists of individual price series for different market locations, and each price series may have different data support. This makes any single-step imputation for all missing entries difficult. First, the individual market-level price time series are imputed using a structural time series model. The individual price series are then aggregated into a single commodity specific country trend, using an exponentially weighted average based on the amount of actual observations (giving exponentially more weight to the less imputed price series). The market-level price series that have sufficient data to carry out univariate modeling, are then imputed using a Generalized Additive Model that uses the commodity specific country trend as predictor of the observed data. Remaining market-commodity pairs without enough data support, are finally spatially interpolated using inverse distance weighting. Individual market-level price time series that do not have enough data support for modeling, or no observations at all, can be initialized by calculating the _K_ -nearest market locations using a Guassian kernel around market coordinates and observed price data. The market-level price time series of these _similar_ markets can then serve as proxies. This patter-matching idea is detailed by Andree (2022). Using these interpolations, the application then adds a homoskedastic disturbance drawn from a Gaussian distribution with a standard deviation of 5% in price levels for each initialization.

> 2In many countries, the exchange rates from official or public sources and available from for instance the European Central Bank (ECB) or Yahoo, are nearly identical and reasonable to use in conversions. In some low-income countries, the institutional rates are not reflective at all of actual retail exchange rates. In such countries, some effort is needed to construct reasonable exchange rate data from country-specific sources. In this paper, daily rates from the ECB finance were used and aggregated to monthly data using averages. We then obtain monthly parallel market rates available from FAO through the FMPA, correct the monthly aggregated official rates using a linear regression to correct for the typical spread that retail consumers face, and in the result replace observations with the FMPA data where available. The result
_ANDR EE_<sup>_ ́_</sup> _AND PAPE_ _SEPTEMBER 5, 2023_ _8_ The model (2) is estimated multiple times as the imputation process continuous to iterate over the data. Andr ́ee (2021) noted that the statistical properties of the data change throughout the imputation process, and that the ideal prediction model grows in flexibility as the process iterates. Specifically, in the sequence of models _{f_ 1<sup>_A, f_</sup> 2<sup>_A, ..., f_</sup> _I_<sup>_A}_for</sup><sup>_I_iterations,themodel</sup><sup>_f_</sup> _i_<sup>_A_</sup> may train on better data than model _fi_<sup>_A_</sup> _−_ 1<sup>andthusthecomplexityofthelearnermaygrowwiththedata.</sup> The application here performs the first half of iterations (set at 4) using an elastic net model (implemented by Hastie et al. (2021)). The penalties in the elastic net model help reduce the impact of uninformative predictors (Friedman et al., 2010). At iterations 5 through 8, a cubist regression is used following Quinlan (1992); Witten et al. (2016).

Cubist is a piece-wise linear model that combines decision trees, boosting, and neighborhood smoothing, that together allow the model to capture several features typical to spatial time series data, including smooth nonlinearities (Andr ́ee et al., 2019), spatial regime switches (Andr ́ee et al., 2017; Andr ́ee, 2020), as well as threshold nonlinearities (Tong, 2015). This makes the cubist nonlinearities model more suitable for numerical data than Random Forest-type nonlinearities (Kuhn et al., 2012).<sup>3</sup> The hyper-parameters of both models are tuned at each step. The modeltuning focuses on a Normalized Mean-Absolute-Error criterion, which is robust to outliers. The elastic net model optimizes over the standard _L_ 1 and _L_ 2 penalties and the mixing parameter (commonly denoted _α_ ). _L_ 1 penalizes likelihood by the absolute sum of coefficients, and _L_ 2 by the sum of squared parameter values, thereby discouraging large parameter estimates but having very different impacts when redundant parameters approach 0. Since exchange rates may suddenly drop (it is not uncommon for currencies in low-income countries to lose a peg), the institutional exchange rate is excluded from the linear elastic net model.

The decisive factor to use Cubist, and not say XGBoost, is that Cubist models are very fast to tune and validate. This is of critical importance to the application. In the current application, a relatively large number of food items is processed, and narrow set of tuning parameters is preferred over a wide one to make the computation feasible. The cubist model tunes over the neighborhood size used for smoothing, and the boosting iterations, using grid of _Neighborhood_ = (4 _,_ 6) _× Committees_ = (10 _,_ 25) combinations. Models are validated using _k_ = 4 folds, this means that _k_ + 1 _× Neighborhood × Committees_ = 20 model specifications provides an estimate of typical retail exchange rates.

> 3Cubist is an extension of M5 regression trees that incorporates pruning, neighborhood smoothing and boosting. Essentially is uses a computationally efficient strategy to recursively partition the data space and fit simple piece-wise linear prediction models within each partition, whose predictions are combined using neighborhood averaging of local model predictions. The advantages over M5 are that it can produce smoother transitions across numeric outputs, and much faster runtime. Both being of high importance to the current application. The advantages over Random Forest are that the cubist model has linear regressions at terminal nodes and so it can extrapolate slightly out of range, while Random Forests can only interpolate using medians or averages of typical values associated within the ranges of the input data.
_MACHINE LEARNING AND HIGH-FREQUENCY SURVEYS IN PAPUA NEW GUINEA 9_ need to be produced at each imputation step. For 20 food items, 8 iterations, and 5 imputations, the total number of models involved is 16,000, highlighting the need for fast model building.

# _C. Validation and optimization_

Since the food price data is incomplete, the true inflation of the food basket is never observed. This makes validation against true data particularly difficult. Cross-validation techniques are used to assess the predictive accuracy of the many individual price prediction models involved at the level of individual prices.

Since there are many food items (here _d ∈_ 1 _, ..., D_ ), each predicted by an ensemble of multiple models (here _m ∈_ 1 _, ..., M_ ), the robustness of the final estimate for the overall food basket imputation is summarized in a single score that summarizes the prediction accuracy of all models involved. First, a normalized MAE for the log price of food item _d_ is constructed as the ratio of the MAE of the model for that food item to the MAE obtained by a simple mean prediction. Since each MAE estimate represents an average point percent error rate due to the log nature of the price data, the individual MAE values are averaged geometrically.

where _MAE_<sup>�</sup> _m,d_ is a cross-validation estimate of MAE using the standard formula for MAE, _MAEd|μ_ ˆ is the MAE calculated using observed data and the sample estimate for the unconditional mean. Since the true data range is not observed, and averaging is known to improve ensemble performance, the quantity from equation 3 is likely a conservative estimate of true error.

� The focus next is on the quantity 1 _− NMAE_ , which is the share of the total absolute variation in the demeaned log price data explained by the imputation model. The _D_ values are averaged as follows.

where _Z_ is the Fischer Z-transformation and _Z_<sup>_−_1</sup> its inverse, and _w_ are the relative weights of the price component in the final price index. The final score from equation 4 roughly has the interpretation of the average _R_<sup>2</sup> of the food price index, using a robust calculation of out-of-sample errors (Andr ́ee, 2021).<sup>4</sup>

> 4The Fischer Z-transformation is used because 1 _−NMAE_ � approaches the R-squared when prediction
_ANDR EE_<sup>_ ́_</sup> _AND PAPE_ _SEPTEMBER 5, 2023_ _10_ It is important to note that equation (4) only provides a single validation result to assess overall robustness of the imputed index. It has, however, limited use in assessing the robustness of the price monitor at any given moment in space and time. To track observation-level price uncertainty, we propose a time-varying item- and location-specific trust score that combines validated predictive power with local data availability into a single rating that can be used to gauge reliability of local results. We also estimate time-varying volatility to model typical intramonth price ranges.

# Local accuracy

Recall that the objective is to predict unobserved data and that residuals are therefore also unobserved. This means there is no way of directly measuring observation-level reliability without external validation, which, if possible, voids the need for modeling the observation. The second next best option is to produce an internal validation procedure based on what can be validated and some intuition around what drives uncertainty.

Recall that for any price prediction, some of the covariates (other prices) may also be unobserved. Hence, reliability depends not only on the prediction model for item _d_ but also on the overall predictive power that all models have across food items plus the overall data availability rate when the prediction is made. When data coverage is high, less modeling is needed, and the overall accuracy with which the inflation rate is estimated is higher. When more than the average share of data is observed, the true _R_<sup> ̃2</sup> is likely above the cross-validated one. In the extreme, when the data is fully observed and no prediction is needed, the accuracy is 100%. When no data is available on the other hand, and the data is fully modeled, the _R_<sup> ̃2</sup> is possibly much lower than the cross-validated one. How much depends on the overall difficulty of extrapolating prices based on trends only.

Based on this basic intuition, a trust indicator is constructed that scales the _R_<sup> ̃2</sup> between a low point estimate of accuracy established using a simple trend-only model, and the high-point of 1 when data is fully observed, through _R_<sup> ̃2</sup> at the point where data coverage is average. This requires a basic mapping from the share of data coverage _S_ to a corresponding level for _R_<sup> ̃2</sup> The map _h_ is constructed as follows. Let _R_<sup> ̃</sup> _d_<sup>2:= 1</sup><sup>_−_</sup> _NMAE_ � _d_ denote the pseudo R-squared reflecting average predictive power obtained for food item _d_ estimated using cross-validation, with _R_<sup> ̃2</sup> being the index equivalent. Let _S_ be a vector of length _N × T_ where _N_ is the number of market locations, and _T_ is the length accuracy is high, and so the square root approaches the correlation coefficient between predictions and observations. It is well-known that correlation coefficients cannot be averaged directly, the Fischer Z- transformation is intended to counter the biases introduced when averaging correlation coefficients.
_MACHINE LEARNING AND HIGH-FREQUENCY SURVEYS IN PAPUA NEW GUINEA 11_ of the time series, that stacks the share of complete survey responses across all food items. I.e., when _Si,t_ equals 0 _._ 5 then at location _i_ and time _t_ , a 50% share of the data needed to calculate the basked price was observed. Let _ma_ ( _·, p_ ) be a _p−_ period moving average function and _S_<sup>12</sup> := _ma_ ( _S,_ 12) denote its 12-month moving average so that each entry corresponds to the share of observed data needed to calculate the basket inflation rate. Figures B3 show how the response rates varies over time. Let _P_ ̄ _N_<sup>beasingletimeseriesoflength</sup><sup>_T_,thataverages</sup> the imputed food price index of _N_ markets, and let _SN_<sup>12</sup> ̄<sup>equivalentlybethetime</sup> series that contains the market average of the 12-month moving average data availability rates.<sup>5</sup> Finally, let _SN,d_<sup>12</sup> ̄<sup>betheitem-specificanalog,and</sup><sup>_S_</sup> _N,_<sup>12</sup> ̄ _−d_<sup>bethe</sup> quantity constructed for a basket that excludes item _d_ .

Recall that _R_<sup> ̃2</sup> is the average reliability of the food price index, thus it coincides with the estimated average reliability of _PN_ ̄<sup>evaluatedattheapproximatemean</sup> value of _S_<sup>12</sup> ̄<sup>thelatterbewrittenas</sup><sup>_μS_12</sup> ̄<sup>Toproduceatime-varyingproxy</sup> _N_<sup>,</sup> _N_<sup>.</sup> for _R_<sup> ̃2</sup> that is reflective of reliability when _SN_<sup>12</sup> ̄<sup>isrespectivelyhighorlow,we</sup> need to construct reasonable lower and upper bounds for _R_<sup> ̃2</sup> and a transition path between them across the _S_ plane. The low point _Rlow_<sup>2issetbyfirstcross-</sup> validating a third-order Taylor expansion of a trend component estimated with a small Ridge penalty against the full price index, allowing for fixed seasonal variation captures with dummies.

If the price index evolves following a basic trend with seasonal component, then its _R_ ̃<sup>2</sup> extrapolationis high. If thecanpricebe indexdone reliablyevolves evenwithoutwithouta seasonaldata andtrendthestructure,low pointthenfor the low point of _R_<sup> ̃2</sup> is essentially 0 so that no reliable estimate can be produced without data.

The functional form mapping the data availability to the associated _R_<sup> ̃2</sup> value is constructed by upsampling the vectors _R_<sup>2</sup> := ( _Rlow_<sup>2</sup><sup>_, ..., R_2</sup><sup>_, ...,_1)and</sup><sup>_S_:=</sup> (0 _, ..., μSN_<sup>12</sup> ̄<sup>_, ...,_1)toagridof100elementsandusingasplineinterpolationon</sup> log scale to interpolate the missing entries ( _..._ ), and then constructing a map _h_ : _R_<sup>2</sup> _→S_ using a local exponential smoother. The trust scores _T_<sup> ̃</sup> are generated at the one-digit level using the function:

> 5Since fully observing prices at one market has a higher impact on the ability to model realistic price trends than observing a equal total amount of data distributed across all markets, the shares are averaged using the Root-Mean-Squared to give more weight to the presence of unmodeled data series. This is omitted from notation to avoid further cluttering.
_ANDR EE_<sup>_ ́_</sup> _AND PAPE_ _SEPTEMBER 5, 2023_ _12_ The food-specific scores are generated as: where _hd_ is constructed in the same way as _h_ , but on the grids _R_<sup>2</sup> _d_<sup>:=(</sup><sup>_R_</sup> _low_<sup>2</sup><sup>_∗_</sup> _R_<sup>2and</sup><sup>_S_:=(0</sup><sup>_, ..., μS_12</sup> ̄<sup>Notethatduetotheharmonic</sup> _d_<sup>_/R_2</sup><sup>_, ..., R_</sup> _d_<sup>2</sup><sup>_, ...,_1)</sup> _N,d_<sup>_, ...,_1).</sup> averaging of data availability in equation 8, the trust score is always guaranteed to be 10 when the data is fully observed even when no data on the covariate side is available.<sup>6</sup> Values below 6 are dis-satisfactory, indicating that the monitor is dysfunctional. Values of 6 _−_ 8 are in the moderate to good range, and values 8 and above imply that price-tracking is very reliable, and values above 9 imply that the impact of any missing data is unnoticeable.

# Intra-month price ranges

Since there is substantial intra-month volatility that is hidden by static monthly price quotes, the price-level estimates are accompanied by intra-month price range estimates. The aim is to be able to compare price changes to their typical shortrun variations, so to put the significance of change into context. Broadly speaking, a 10% change in basket prices would be a significant rate of inflation if the typical monthly variation in prices is less than a percent, but negligible when that typical monthly variation in price is itself 10%. As the application will reveal, fresh produce prices in PNG are particularly volatile so the intra-month price range estimates provide important insights about the price dynamics.

The price ranges are estimated as an Open-High-Low-Close time series object which is defined as: where _P_ is the imputed price series and E∆ _α_ is the expected change in the _α_ - percentile cases. The combined results can be plotted on a candle chart with the majority of price action to have occurred within the body of the candles and wicks indicating the price range between the average of the highest 50% of intra-month prices, and the average price of the lowest 50% of intra-month prices.

The first three quantities in equation 9 are estimated by modeling the time-

> 6In the same way, if the covariates are fully observed but the modeled food item has missing values, the trust score would be 10. This situation is extremely unlikely to ever occur due to the nature of the data.
_MACHINE LEARNING AND HIGH-FREQUENCY SURVEYS IN PAPUA NEW GUINEA 13_ varying distribution of the month-on-month inflation process as an autoregressive moving average process with fractionally integrated generalized autoregressive conditional heteroskedasticity (ARMA-fiGARCH) following Baillie et al. (1996). The estimation is detailed in the Appendix.

# **III. Application to Papua New Guinea’s Fresh Produce Prices**

With a population of over 11.87 million people (2021), PNG is the largest and most populated island state in the Pacific. Yet, the country is known to be one of the most data-scarce and impoverished in the world. The World Bank’s last poverty assessment was performed well over a decade ago in 2009 and estimated that 38% of the population lived under the US$1.90 poverty line at the time. The country performs poorly on a number of social development indicators, but accurately tracking development trends remains difficult because of a lack of representative primary data (Edmonds et al., 2018).

In PNG, the traditional consumer price index (CPI) is produced at an aggregate level, using data from a few urban areas. This does not properly reflect the majority of the population as the World Development Indicators estimate that 87% of PNG’s population lives in rural areas (2021). Traditional price data collection also aims to follow a deliberate sampling and measurement process that is not well suited for monitoring during crises situations, when price levels may rise and fall rapidly. For instance, at the time of writing, the latest CPI for PNG lags by 6 months.

Scarce analyses point out that food markets in PNG are likely to be poorly integrated and disrupted by recent macro-shocks (Tracer, 2005; Huffaker et al., 2021; Davila et al., 2021), and as such, high volatility due to local shifts in supply and demand can be expected. With 87% of the population in rural areas, and highly localized price dynamics, a local-area CPI would be needed to more adequately describe price trends in different rural or poverty-stricken areas, where the majority of the population resides in fragile situations.

The paper uses end-of-July data available from IFPRI, downloaded from the Papua New Guinea Fresh Food Price Monitoring Tool on August 17, 2022. The data reports monthly prices in Kina (local currency) as measured in 8 different market locations throughout the country (Banz, Goroka, Kokopo, Kundiawa, Lae, Madang, Mt. Hagen and Port Moresby). Raw price data dates back to mid-2009, but there are periods of substantial data gaps, particularly between 2016 and 2018. In total, the data contains 79 unique food items, but most do not have a sufficient number of observations to create a time series.

Since the focus of the paper is on monitoring price inflation, a basket of food items with data coverage over a long time period is essential. The application also aims to understand what the right design may look like for future co-deployment of rapid survey systems and machine learning imputations, and so different data selections are used to investigate the trade-offs in data availability and survey coverage. Three baskets are selected. A 25-item version since 2009 and 2014 that
_ANDR EE_<sup>_ ́_</sup> _AND PAPE_ _SEPTEMBER 5, 2023_ _14_ seeks to track inflation over a long time period using a broad-as-possible food basket, a 25-item version that starts in 2017 that tracks a narrower basket of items that have better data coverage. There are minor item differences in the basket specification and data coverage, see Table 2.

Table 2—Summary of raw food price data in Papua New Guinea.

||Coverage|12-month|Items|Markets|
|---|---|---|---|---|
|2009-2022|36.96%|50.71%|25<sup>_∗_</sup>|8|
|2014-2022|29.17%|52.17%|25<sup>_∗∗_</sup>|8|
|2017-2022|39.01%|55.47%|27<sup>_∗∗∗_</sup>|7|
|FCS average|51.47%||7|27|

_Note:_ The first column reports the study period, the second column reports the data coverage across the full study period, the third column reports the data coverage in the last 12 months, the last columns reports the number of markets modeled out of the total number of markets included in the data snapshot. _∗_ Basket consists of 1 unit of each: _Aibika, Amaranthus-Aupa, Banana-Cooking, Banana-Ripe, Broccoli, Cabbage-English, Capsicum, Carrot, Cassava, Choko-Tips, Cucumber, Ginger, Lemon, Lettuce, Mandarine, Onion Bulb, Orange, Pawpaw, Peanut, Pineapple, Sweet Potato, Pumpkin-Tips, Taro True, Tomato, Watermelon. ∗∗_ Basket consists of 1 unit of each: _Aibika, Amaranthus-Aupa, Banana-Cooking, Banana-Ripe, Broccoli, Cabbage-English, Capsicum, Carrot, Cassava, Choko-Tips, Cucumber, Ginger, Lemon, Lettuce, Onion Bulb, Orange, Pakchoi, Pawpaw, Peanut, Pineapple, Sweet Potato, Pumpkin-Tips, Taro True, Tomato, Watermelon. ∗∗∗_ Basket consists of 1 unit of each: _Aibika, Amaranthus-Aupa, Banana-Cooking, Banana-Ripe, Broccoli, Cabbage-English, Capsicum, Carrot, Cassava, Choko-Tips, Cucumber, Fern, Ginger, Lemon, Lettuce, Onion Bulb, Orange, Pakchoi, Pawpaw, Peanut, Pineapple, Sweet Potato, Pumpkin-Tips, Taro True, Tomato, Watermelon, Wongbok. Source:_ The statistics have been prepared by the author for this paper based on end-of-July (2022) food price data from IFPRI. The FCS average has been reproduced from table A1 in Andr ́ee (2021) who presented identical estimates for 25 FCS countries (excluding PNG).

The first selection tries to optimize for what benefits historical studies and the key challenge is the lower data availability, the third selection tries to optimize for monitoring purposes of current trends and the key challenge is the shorter time dimension which makes trend modeling more difficult. In all cases, the foods are all fresh produce vegetables, and the baskets have strong item overlap. The application keeps the item weights fixed at 1 KG of each item when constructing the basket price. The resulting price index is a Laspeyres index that does not seek to adjust for changing consumption behavior that could in part be induced by relative price rises. This is appropriate in an extreme-poverty or food-insecurity context in which food accounts for the majority of household expenditures and shifts toward less nutritious foods may keep basket prices equal and at the same time create food and nutrition security risks.

Taken together, the data selection simulates the real-world challenge: when a new survey program is deployed, it is possible to design it so that a high number of food items will be tracked but the data will naturally be limited in temporal scope. As the program carries on for longer, it typically becomes more difficult to ensure that the full basket remains observed and so the food basket consisting of items with “good data coverage” narrows. Finally, over long time periods,
_MACHINE LEARNING AND HIGH-FREQUENCY SURVEYS IN PAPUA NEW GUINEA 15_ possible temporal overlap between a higher number of food items may again be exploited to estimate (“stitch together”) a complete basket price over a long time period, albeit with a higher share of missing data.

When compared to the average FCS country considered by Andr ́ee (2021), the PNG data stands out in the sense that it has more food items but fewer markets. The data availability is below the average of FCS countries considered by the application presented by Andr ́ee (2021).

# _A. Validation results_

Table 3 summarizes the cross-validation results along with basic aggregate statistics calculated from the imputed Food Price Index. Compared to results obtained in other FCS countries, the annualized inflation rates, maximum drawndown, and annualized volatility are similar to what has been observed for staple food prices in comparable countries. The annualized volatility is slightly higher, but not by a standard deviation. higher. The _R_<sup> ̃2</sup> here is more than a standard deviation below the FCS median. Nevertheless, it is within the range of FCS results (the results presented for Chad reached 0.69, and 0.75 for Central African Republic). The results therefore show that the new approach reaches comparable results even with lower data coverage and a much higher number of food items.

Table 3—Summary of raw food price data in Papua New Guinea.

||Annualized|Annualized|Max.|Avg. monthly|Avg. annual|CV-|
|---|---|---|---|---|---|---|
||Infation|Volatility|Drawdown|market corr.|market corr.|score|
|2009-2022|6.21%|15.33%|-21.28%|0.18|0.33|0.71|
|2014-2022|4.9%|11.83%|-19.08%|0.26|0.57|0.76|
|2017-2022|6.5%|12.4%|-18.35%|0.24|0.35|0.77|
|FCS median|4.92%|11.83%|-24.17%|||0.87|
|FCS St.Dev|13.02|5.10|10.36|||0.072|

_Note:_ The first column reports the study period, the second column reports the data coverage across the full study period, the third column reports the data coverage in the last 12 months, the last columns reports the number of markets modeled out of the total number of markets included in the data snapshot. _Source:_ The statistics have been prepared by the author for this paper based on end-of-July (2022) food price data from IFPRI. The FCS median and St.Dev have been calculated from table A4 in Andr ́ee (2021), who present identical estimates for 25 FCS countries (excluding PNG).

The average food price inflation rates are in a reasonable range. For instance, the annualized official CPI price inflation rate for the full 2009-2022(Q2) period was 5.33%. Note that a direct comparison with food price inflation would be interesting, but would have to account for the differences in basket specification.

Considering the high price volatility, two to almost three times annualized inflation, it is also reasonable to assume that any direct measurement of prices would face difficulties with precision. For instance, consider 12 _._ 4%<sup>1</sup><sup>_/_12</sup> = 1 _._ 23% as a typical error in monthly price measurement of a single item that stems from natural volatility. That level of error is almost 20% of a 6.21% inflation rate (the
_ANDR EE_<sup>_ ́_</sup> _AND PAPE_ _SEPTEMBER 5, 2023_ _16_ long-run averages estimated here) and over 20% of the official 5.33% long-run average. The lowest imputation _R_<sup> ̃2</sup> of 0 _._ 71 introduces only an additional 12% measurement accuracy compared to such a golden standard, which is not negligible but suggests that the real-time imputation of price surveys produces a viable alternative to official statistics, particularly when the timeliness and costs of both approaches are pitted against one another.

Another result that is worth commenting on is that the average correlation between the market-level food price indexes is low. At a monthly level the correlation is only around 0.18-0.26, and 0.33-0.57 when the price data is annualized. This means that there is on average little price transmission over short time spans. In this setting of weakly integrated markets, a single CPI index may not be representative of actual location-specific price dynamics.

The table provides some indication that imputation accuracy goes down when covering longer time periods. This may on the one hand be driven by increased price heterogeneity in the past, as indicated by the lower cross-market correlations, or by lower data availability as indicated by table 2. It is hard to gauge from the basic results whether there is erosion in performance taking place, which is part a reason why the trust score from section II.C is useful.

# _B. Prediction results_

The graph in figure 1 shows the modeled food price index using the 2017-2022 data selection, along with the obtained inflation rates and the trust scores. Graphs for the 2014 and 2009 starting points are available in the appendix.

The results show that for each data selection, a similar inflation rate of approximately 17% is obtained for the final period. A 1% difference is sensible since the food baskets are different. All three results also highlight a sharp price rise, with inflation reaching well above 20%, during 2019. In 2017-2018, there is a hiatus in the data collection process, and the trust scores in the three charts clearly highlight this. In the long-run chart (2009-2022), the trust score remains in the acceptable 6+ range. It is also clear that when zooming out, the data gap occurs in a period of reasonable price stability.

Even though the 2009 version has the largest share of imputations, it is worth pointing out that the basket result paints a similar picture for the overall trend as the less modeled 2014 version and more accurate 2017 version. In all cases, the prices move largely sideways from 2014 through 2018, before surging into 2019 and falling rapidly into 2020, and resuming on an upward path. This suggests that for broad monitoring purposes, the procedure produces consistent results across all data dimensions. In conclusion, the system operates well and is able to generate useful country-level monitoring results that are robust to changes in the data collection process.
_MACHINE LEARNING AND HIGH-FREQUENCY SURVEYS IN PAPUA NEW GUINEA 17_ <!-- Start of picture text --> Papua New Guinea − Food Price Index (LCU, 2018 = 1) 2017−01−01 / 2022−07−01<br>1.50<br>1.45<br>1.40<br>1.35<br>1.30<br>1.25<br>1.20<br>1.15<br>1.10<br>1.05<br>1.00<br>Price Inflation (Year on year, %) 16.01<br>30 30<br>20 20<br>10 10<br> 0 0<br>−10 −10<br>−20 −20<br>Trust Level 8.5<br>10 10<br> 8 8<br> 6 6<br> 4 4<br> 2 2<br>2017 2018 2019 2020 2021 2022<br>Jan Jul Jan Jul Jan Jul Jan Jul Jan Jul Jan Jul<br>2017 2017 2018 2018 2019 2019 2020 2020 2021 2021 2022 2022<br><!-- End of picture text --> Figure 1. Estimated food price dynamics in PNG, 2017-2022.

_Note:_ Food basket price in local currency, average across markets, January 2018 = 1. The candle-chart on top shows the estimated Open, High, Low and Close prices for the month. Red candles indicate months where end-of-month prices are lower than start-of-month prices. The size of the candle is an indicator of intra-month price variation. Along with estimates of prices, the shows the 12-month moving averages as a dotted line, as well as Bollinger Bands in the grey shaded area, which can help assess whether prices are high or low, and whether prices increases are sharp or smooth, on a relative basis. The middle chart shows the 12-month percentage change rate in prices, as a measure of inflation. The trust score at the bottom is a metric that factors in both data availability and accuracy of imputations to express confidence in the estimates on a scale of 1-10. _Source:_ Figure prepared by the authors for this paper.
_ANDR EE_<sup>_ ́_</sup> _AND PAPE_ _SEPTEMBER 5, 2023_ _18_ <!-- Start of picture text --> Papua New Guinea − Unofficial exchange rate (Parallel−market Estimate) (1 USD/LCU) 2009−01−01 / 2022−07−01<br>3.4<br>3.2<br>3.0<br>2.8<br>2.6<br>2.4<br>2.2<br>2.0<br>1.8<br>Price Inflation (Year on year, %) 0.57<br> 30 30<br> 20 20<br> 10 10<br> 0 0<br>−10 −10<br>−20 −20<br>2009 2010 2011 2012 2013 2014 2015 2016 2017 2018 2019 2020 2021 2022<br>Jan Jan Jan Jan Jan Jan Jan Jan Jan Jan Jan Jan Jan Jan<br>2009 2010 2011 2012 2013 2014 2015 2016 2017 2018 2019 2020 2021 2022<br><!-- End of picture text --> Figure 2. Estimated parallel-market exchange rate dynamics in PNG, 2009-2022.

_Note:_ Cost of USD in local currency, average across markets. TThe candle-chart on top shows the estimated Open, High, Low and Close prices for the month. Red candles indicate months where end-ofmonth prices are lower than start-of-month prices. The size of the candle is an indicator of intra-month price variation. Along with estimates of prices, the shows the 12-month moving averages as a dotted line, as well as Bollinger Bands in the grey shaded area, which can help assess whether prices are high or low, and whether prices increases are sharp or smooth, on a relative basis. The middle chart shows the 12-month percentage change rate in prices, as a measure of inflation. The trust score is not displayed since it is 10 everywhere. _Source:_ Figure prepared by the authors for this paper.
_MACHINE LEARNING AND HIGH-FREQUENCY SURVEYS IN PAPUA NEW GUINEA 19_ <!-- Start of picture text --> Mt. Hagen − Cabbage−English (1 unit) 2009−01−01 / 2022−07−01<br>6<br>5<br>4<br>3<br>2<br>1<br>0<br>Price Inflation (Year on year, %) 50.41<br>500 500<br>400 400<br>300 300<br>200 200<br>100 100<br> 0 0<br>−100 −100<br>Trust Level 8.4<br>10 10<br> 8 8<br> 6 6<br> 4 4<br> 2 2<br>2009 2010 2011 2012 2013 2014 2015 2016 2017 2018 2019 2020 2021 2022<br>Jan Jan Jan Jan Jan Jan Jan Jan Jan Jan Jan Jan Jan Jan<br>2009 2010 2011 2012 2013 2014 2015 2016 2017 2018 2019 2020 2021 2022<br><!-- End of picture text --> Figure 3. Estimated price dynamics for cabbages in PNG, 2009-2022.

_Note:_ Price of cabbages in Mt. Hagen in local currency. The candle-chart on top shows the estimated Open, High, Low and Close prices for the month. Red candles indicate months where end-of-month prices are lower than start-of-month prices. The size of the candle is an indicator of intra-month price variation. Along with estimates of prices, the shows the 12-month moving averages as a dotted line, as well as Bollinger Bands in the grey shaded area, which can help assess whether prices are high or low, and whether prices increases are sharp or smooth, on a relative basis. The middle chart shows the 12-month percentage change rate in prices, as a measure of inflation. The trust score at the bottom is a metric that factors in both data availability and accuracy of imputations to express confidence in the estimates on a scale of 1-10. _Source:_ Figure prepared by the authors for this paper.
_ANDR EE_<sup>_ ́_</sup> _AND PAPE_ _SEPTEMBER 5, 2023_ _20_ The graph in figure 2 shows the processed exchange rate data constructed from FAO FMPA parallel market quotes and daily rates from the ECB. The result can be used to dollarize the food price estimates. It is important to note that in many countries with strongly depreciating currencies, the exchange rate may correlate very strongly with food prices. In PNG, the data largely follows a different pattern. From the graph we can infer that the dollar appreciated roughly 40% against the Kina between 2009 and 2022 while food prices more than doubled. Over the long run, food has thus increased not only in local currency but also in dollar terms.

When price monitoring focuses on a single food item, however, it becomes more important to ensure that the individual price series have some degree of data support throughout the entire monitoring period. For instance, the results in figure 3 show an example of cabbage prices in Mt. Hagen, the third largest city and capital of the Western Highlands Province that is located in the large fertile Wahgi Valley in central mainland. The result highlights that in the PNG survey data, there are considerable data gaps during which the trust score dips severely. The chart also shows that there was a major price shock that resulted in triple digit product-specific inflation rates in the area. Such large price shocks are less likely to occur in countries where markets are more integrated, and highlight the importance of local price monitoring in places such as PNG where they are not.

# **IV. Discussion and conclusion**

While many countries globally are currently experiencing some of the highest inflation rates in decades, the Pacific Islands lack good capabilities to track food price developments in near-real-time. However, without such capabilities, it is difficult to monitor the situation and inform policies in a timely manner, which is particularly crucial in such a volatile environment.

To gain near-real-time insights, this paper applied a recently developed machinelearning approach to impute local market prices for fresh produce in local markets in PNG. The aim is to investigate the feasibility of monitoring food price inflation continuously relying only on incomplete and intermittent local market-level survey data that can be gathered in a rapid fashion. The application to fresh produce prices in PNG is made particularly challenging by high intra-month price volatility, low cross-market price correlations, and weakly defined food-specific price trends.

The paper runs cross-validations for imputation strategies under different designs in terms of numbers of markets, food items and time periods covered. The results show that imputations can achieve accuracy that is attractive when compared to costly direct measurement of prices. For instance, the imputations processed 27 food items across 8 markets and explains approximately 71% (20092022) to 77% of (2017-2022) of variation in out-of-sample data. When natural price volatility is taken into account, it is likely that this approach only sacrifices a little over 10% of accuracy when compared to direct measurement. Consider-
_MACHINE LEARNING AND HIGH-FREQUENCY SURVEYS IN PAPUA NEW GUINEA 21_ ing this, the low-cost of imputations, and the major gaps in primary data in the Pacific Islands, these results suggest that the methods could be deployed in the Pacific Island region to provide unique insight into local price dynamics.

The methods have wider applicability and could help to fill crucial data gaps in data scarce regions such as the Pacific Islands, especially in conjunction with specifically designed continuous surveys. Potential indicators include tracking broad price indices, but possibly the same strategies could be used for other development indicators. Sub-national estimates of prices may also improve smallarea estimates of poverty and welfare, which is an important topic actively being explored by Lange et al. (2021) and Newhouse et al. (2022).

The encouraging results set the stage for a real-time evaluation of the methods along with external validation. Below are a few directions for future efforts.

First, the high intra-month volatility in prices suggest that supply, demand and prices are fluid in PNG markets and economically meaningful price action may be observed in the higher frequency domains. The application showed that it worked well with a short time series of data gathered from 2017 onward. This suggests that a system that gathers data at a weekly interval may also work if the data exhibit meaningful variation. Similarly, a daily interval approach may generate enough survey data to process and yield results within weeks or months after deployment. Such high frequency systems may be particularly valuable to monitor trends after natural disasters. One approach could be to maintain such systems at low cost by sustaining a limited data gathering operation, but quickly ramp up data gathering during periods of heightened interest such as when inflation is high or once a disaster has emerged and intense price action is anticipated.

Market price data gathering can be inexpensive if enumerators can reach multiple markets in a short time. However, geographically highly dispersed markets as in many Pacific Islands can make such an effort unfeasible. At the same time, phone surveys are increasingly used to collect household information across wide geographical areas, but at low cost. Continuous household-based phone surveys can create an opportunity to collect a variety of prices across large areas at almost no additional cost. Systematically assigning different items across households can create exactly the kind of data set with systematically missing items, which the proposed algorithm takes as an input to impute price indices. Thus, merging the efforts of household-based data collection via phones with price data collection is promising. However, the impact of challenges like measurement error from reporting prices through households, as well as an implicitly biased sample of markets (which households with phones use) will need to be carefully evaluated.

Second, the number of markets covered by the application was relatively low when compared to earlier related efforts in FCS countries. Moreover, correlations in prices across markets was very low in PNG. On the one hand this suggests that inflation tracking in PNG may benefit from data gathering at additional markets. On the other hand, the fact that the imputation results were still reasonably ac-
_ANDR EE_<sup>_ ́_</sup> _AND PAPE_ _SEPTEMBER 5, 2023_ _22_ curate suggests that the approach may be viable in other settings where markets are only weakly interlinked. It is important to note that the individual Pacific Islands share common trade links and are heavily reliant on the same imported foods (Snowdon et al., 2013). Thow et al. (2011) analyze the effects of trade policies in the region and observes that from 1960 to 2005, liberalization policies, export promotion, protection of the domestic meat industry and support for foreign direct investment, have contributed to a reduced availability of traditional staples and increased availability refined and processed foods. Such commonalities suggest that prices across islands may be weakly interlinked, and so a system that monitors prices through the combination of surveying and imputing may work across islands. A joint monitoring system would be economically more attractive to maintain for an international institution than separate systems for each islands, which may quickly be cost-prohibitive on a per-capita basis.

Third, the low correlation in prices between markets suggest that local PNG markets are not basic price-taking components of a wider market. This provides a unique opportunity to study the impacts of local supply shocks on prices. In the more common situation in which a country imports food and is price-taking, local supply does not affect prices as shortfall is offset by increased imports. The presence of strong local price-supply correlations may provide an opportunity to validate other models that seek to estimate dynamics in agricultural yield.

Finally, the extremely high item-specific price volatility suggested that future price monitoring efforts should prioritize high primary data coverage in staples that are economically and nutritiously important. Charlton et al. (2016) review 29 studies on primary food sources for Pacific Islanders and emphasizes that Pacific Islanders depend for large parts of their household income and direct food consumption on fishing, a practice that has come increasingly under threat by climate change (Barnett, 2011; Connell, 2015). From this perspective, tracking catch prices or daily labor rates in the fishing industry may be useful to consider in the future. For all areas, tracking processed foods, that are generally imported from shared origins, would be helpful.

# **Acknowledgments**

The authors thank Michael Wild for comments on an earlier version of this work, Emily Schmidt and Soonho Kim at IFPRI for data support, and Andres Chamorro, Lydia Kim, and Kamwoo Lee for project support.

The findings, interpretations, and conclusions expressed in this paper are entirely those of the authors. They do not necessarily represent the views of the World Bank and its affiliated organizations, or those of the Executive Directors of the World Bank or the governments they represent.

# **Funding statement**

The country application was prepared to support the Pacific Observatory, a wider initiative aimed at providing non-traditional data sources as complements
_MACHINE LEARNING AND HIGH-FREQUENCY SURVEYS IN PAPUA NEW GUINEA 23_ to official statistics for improving the frequency, timeliness, and granularity of key economic/development indicators for data-driven policy-making in Papua New Guinea and the Pacific Islands. Funding from DFAT (TF0B6892 and TF0B6579) is gratefully acknowledged.

Additional funding by the Federal Ministry for Economic Cooperation and Development (BMZ, Germany) as part of the World Bank’s Food Systems 2030 (FS2030) Multi-Donor Trust Fund program (TF073570 and TF0C0728) is gratefully acknowledged. The methodological improvements suggested in the paper have been implemented in the live version of the Real-Time Food Prices (RTFP) data set on the World Bank’s MicroDataLibrary as part of the Global Food and Nutrition Security PASA activities financed under the FS2030 grants.

The live data set can be accessed here https://doi.org/10.48529/2zh0-jf55 and here https://doi.org/10.48529/5hgz-q149. The improved speed of the new routines has allowed enabling the processing of other non-food items available in the original surveys as additional predictors.

# **Code and data availability statement**

Access to the code (Price Monitor, Version: 842) used to produce the results of this paper can be requested. Training data and updated validation statistics similar to those used in the paper, are published along with the model outputs as part of the RTFP data.