_ANDR EE_<sup>_ ́_</sup> _DECEMBER - 2021_ _14_ The training data contains in addition to the observed prices a small draw of 10% of the imputations generated from the previous regression estimates. These synthetic data points help balance prediction performance in thinner regions of the sample space. When the training data consists only of actual observations, the predictions may be biased toward purely observed value ranges. Adding synthetic cases using previous predictions makes the estimation problem slightly more representative of the missing data. The validation sample always consists only of actual data points, thus excluding imputations.<sup>21</sup> The model-tuning focuses on a Normalized Mean-Absolute-Error criterion. The reason for this criterion is that the MAE is less impacted by outliers than the common Root-Mean-Squared Error measure and has a stable interpretation across applications.<sup>22</sup> Note that an attempt will be made to replace outliers by missing data values and imputing them, but there is no guarantee that all outliers are captured and so an outlier-robust prediction validation metric is a safe option. There are _M_ validation results for each food item, but the interest is primarily in the robustness of the final imputed price index which combines all the predictions. The validation results are condensed for better presentation. In particular, a cross-comparable confidence score is constructed from the individual validation estimates.<sup>23</sup> The metric is defined as follows. First, a normalized MAE for food item _d_ is constructed as the ratio of the MAE of the model for that food item to the MAE obtained by a simple mean prediction. Since each MAE estimate represents an average point percent error rate due to the log nature of the price data, the individual MAE values are averaged geometrically.

where _MAE_<sup>�</sup> _m,d_ is a cross-validation estimate of MAE using the standard for-

> 21Note that throughout the simulation, the quality of synthetic training data improves. Such techniques are also being explored elsewhere. For example, Lee and Braithwaite (2020) use a regression chain of image recognition models and feature-based models that update one another’s training data which improved their learning results. In the current paper, _f_ is parameterized using a piece-wise linear approach and adding bootstrap draws from previous imputations helped improve validation performance on actual data by allowing the models to train on denser representative example data near the edges of the sample space. This helps stabilize extrapolations outside of the sample space.

> 22With simple arithmetic one can show that _MAE ≤ RMSE ≤_<sup>_√_</sup> _<u>nMAE</u>_ , which reveals that the upper-limit of RMSE varies with sample size and has different interpretations across applications.

> 23 _M × D × I_ cross-validation metrics can be computed in each country application. The result at iteration _I_ is used to diagnose the quality of imputations, but the full validation sequence provides a diagnostic to determine a relevant value for _I_ . In particular, throughout the simulation, the crossvalidation metrics should improve. The number of iterations for stochastic imputation is often determined by observing whether the means and variances of the predictions start changing in a purely random fashion. In the current case, the stopping criterion that the cross-validation performance does not improve further is a slightly less vague criterion. The multiple imputation implementation of van Buuren and Groothuis-Oudshoorn (2011) has default values of _I_ = _M_ = 5. In the current application, diagnosing the performance indicated _I_ = 8 was usually sufficient. _M_ = 5 was kept due to the high computational load of the application.