the training dataset. We use proportions of 10, 20, 33, 25, 50, 67, and 75 percent of the crop-cut measures, with the rest of the sample serving as the test set in each scenario. These simulations are performed using the EACI 2017 dataset, which has the largest sample of crop-cut plots.

To assess the robustness of statistical analyses using imputed crop yields, our approach involves several key steps. First, for the within-survey imputation procedure, we train the ML model on one-third of the plots that have crop-cut data and test the model on the remaining two-thirds. This allows us to evaluate the effectiveness of the model in accurately imputing yields within the same survey. Secondly, for the survey-to-survey imputation procedure, we train the machine learning model on the entire crop-cut sample from EACI 2017 and then test its performance on the crop-cut sample from EACI 2018. This step further our understanding of how well the model can generalize and impute data across different survey years. Lastly, we invert the training and test samples to experiment with backcasting. In this process, we train the model on the EACI 2018 crop-cut sample and test it on the EACI 2017 sample. This reverse application provides insights into the model’s capability to predict past yields based on more recent data, further testing the limits of the imputation model’s robustness.

# **4.2 Imputation framework**

## **_Using machine learning (ML) for the imputation model_**

In recent years, ML techniques have increasingly been incorporated into economics research, demonstrating significant utility in various applications. As noted by Athey (2018), ML methods have shown remarkable success in predictive problems, much like the one we address in our study. Introducing ML into agricultural economics enables the modeling of key variables, such as crop yields, which are influenced by a complex array of factors including soil quality, weather conditions, timing of inputs, and other management choices, often characterized by non-linearities and interactions.

Moreover, ML algorithms are specifically designed to prevent overfitting to the training dataset. This is achieved through techniques like regularization and the empirical selection of tuning parameters. While overfitting is generally not a primary concern in missing data imputation for within-survey applications, this feature of ML algorithms becomes particularly advantageous in survey-to-survey imputation scenarios. Here, the ability of ML to generalize across different datasets without overfitting is a significant benefit, enhancing the reliability and applicability of the imputation results.

One complication in our setting is that the set of covariates we use has a hierarchical structure: we conduct imputation at the plot level, as each crop, even when intercropped, is modeled separately. However, we include covariates at both the household and EA levels. A potential issue with using ML algorithms in a non-i.i.d. setting is that test performance may be overly optimistic. This occurs when selecting covariates that identify household characteristics or EAs that are good at predicting yields without capturing the idiosyncratic details of plots that may affect yield values. This, in turn, would decrease the predictive performance of the ML model at the plot level. To mitigate this issue, we have interacted geospatially extracted characteristics with plot characteristics that make sense, for example, the date of planting and the amount of rainfall per dekad. This is not done systematically for all variables to keep 8