_MACHINE LEARNING AND HIGH-FREQUENCY SURVEYS IN PAPUA NEW GUINEA 3_ survey data, for instance by leveraging high-frequency phone interviews to collect data to inform developmental responses to emergencies (Hoogeveen and Pape, 2019). In practice, a rapid survey system hardly produces complete data, particularly when deployed in difficult settings such as during economic turmoil, conflict, or natural disasters, when it is particularly important to understand possible drastic shifts in economic variables. A great deal of innovation is aimed at the question of how to design such surveys to ensure sufficient response rates, and how to process the data to produce correct estimates (Pape and Wollburg, 2019; Pape, 2021; Khamis et al., 2021).

To overcome some of the challenges with high-frequency surveying, Andr ́ee (2021) developed an approach for real-time imputation of ongoing surveys. Specifically, the paper proposed a matrix-completion algorithm based on multiple machine learning models that simultaneously estimates missing entries using information contained in other responses. This works well when the survey tracks multiple correlated variables, and is specifically suitable to impute a price index, as prices of different goods are typically interrelated. The imputation can be applied in high-frequency settings in which only incomplete and intermittent data can be collected.

The aim of this paper is to continue the line of investigation into the ability of machine-learning techniques to impute ongoing surveys in near-real-time and produce continuous data that yield insights into the evolution of possibly fluid events in data-scarce contexts. The paper focuses on a challenging nowcasting objective. It attempts to track inflation in fresh produce prices at the local market level in Papua New Guinea (PNG) using monthly survey data obtained from the International Food Policy Research Institute (IFPRI). The application is made particularly challenging by high intra-month price volatility in fresh produce items, low cross-market price correlations owing to a lack of overall market integration, and weak price trends. The application cross-validates the imputation strategy under different designs in terms of numbers of markets, food items and time periods covered, and shows that when the survey is well-designed, imputations can achieve accuracy that is attractive when compared to costly– and logistically often infeasible–direct measurement. The localized statistics are shown to provide a new granular view on recent food price inflation dynamics in PNG leading up to and after the outbreak of the pandemic, and more recently the conflict in Ukraine.

The application builds on the original algorithm described by Andr ́ee (2021) but suggests methodological improvements that produce faster and more accurate results, particularly in lower data availability settings. The reduced computing time enables the paper to process a higher number of food items. The resulting estimates cover up to 27 fresh food items, across 8 markets, for the period from mid-2009 to July 2022. This is up from an average of 7 and maximum of 16 price items processed by Andr ́ee (2021). The paper also shows that the estimation methods can be applied to different price surveys, including those from country