metadata: authors: 0: author: Yacoubou Djima,Ismael
1: author: Kilic,Talip
count: Mali
docna: 0: docna: Survey Measurement Errors and the Assessment
            of the Relationship between Yields and Inputs in Smallholder
            Farming Systems : Evidence from Mali
docty: Policy Research Working Paper
subtopic: Crops and Crop Management Systems,Climate Change and Agriculture,Food Security,Gender and Development,Labor & Employment Law,Agricultural Economics
teratopic: Gender,Law and Development,Agriculture
abstracts: cdata!: An accurate understanding of how input
            use affects agricultural productivity in smallholder farming
            systems is key to designing policies that can improve
            productivity, food security, and living standards in rural
            areas. Studies examining the relationships between
            agricultural productivity and inputs typically rely on land
            productivity measures, such as crop yields, that are
            informed by self-reported survey data on crop production.
            This paper leverages unique survey data from Mali to
            demonstrate that self-reported crop yields, vis-à-vis
            (objective) crop cut yields, are subject to non-classical
            measurement error that in turn biases the estimates of
            returns to inputs, including land, labor, fertilizer, and
            seeds. The analysis validates an alternative approach to
            estimate the relationship between crop yields and
            agricultural inputs using large-scale surveys, namely a
            within-survey imputation exercise that derives predicted,
            otherwise unobserved, objective crop yields that stem from a
            machine learning model that is estimated with a random
            subsample of plots for which crop cutting and self-reported
            yields are both available. Using data from a methodological
            survey experiment and a nationally representative survey
            conducted in Mali, the analysis demonstrates that it is
            possible to obtain predicted objective sorghum yields with
            attenuated non-classical measurement error, resulting in a
            less biased assessment of the relationship between yields
            and agricultural inputs. The discussion expands on the
            implications of the findings for (i) future research on
            agricultural intensification, and (ii) the design of future
            surveys in which objective data collection could be limited
            to a subsample to save costs, with the intention to apply
            the suggested machine learning approach.
geo_reg_key: 517191,1927320
repnme: repnme: Survey Measurement Errors and the
            Assessment of the Relationship between Yields and Inputs in
            Smallholder Farming Systems : Evidence from Mali
display_title: Survey Measurement Errors and the
            Assessment of the Relationship between Yields and Inputs in
            Smallholder Farming Systems : Evidence from Mali
listing_relative_url: /research/2021/11/33587887/survey-measurement-errors-assessment-relationship-between-yields-inputs-smallholder-farming-systems-evidence-mali
topicv3: Agriculture and Food Security m1326205 435,Crop Production m1326627 174,Land Tenure m1326631 888,Gender m1327621 615,Household Surveys m1504071 2761
publishtoextweb_dt: 2021-11-05T00:00:00Z
dois: 10.1596/1813-9450-9841
historic_topic: Gender,Law and Development,Agriculture
origu: Data Production and Methods (DECPM)
no_of_pages: 62
author_keys: 0: author_key: 1200143
1: author_key: 886702
model_extractions: Policy Research Working Paper 9841 50x2030 Data-Smart Agriculture Series Survey Measurement Errors and the Assessment of the Relationship between Yields and Inputs in Smallholder Farming Systems Evidence from Mali _Ismaël Yacoubou Djima Talip Kilic_ Development Economics Development Data Group November 2021
Policy Research Working Paper 9841

# **Abstract**

An accurate understanding of how input use affects agricultural productivity in smallholder farming systems is key to designing policies that can improve productivity, food security, and living standards in rural areas. Studies examining the relationships between agricultural productivity and inputs typically rely on land productivity measures, such as crop yields, that are informed by self-reported survey data on crop production. This paper leverages unique survey data from Mali to demonstrate that self-reported crop yields, vis-à-vis (objective) crop cut yields, are subject to non-classical measurement error that in turn biases the estimates of returns to inputs, including land, labor, fertilizer, and seeds. The analysis validates an alternative approach to estimate the relationship between crop yields and agricultural inputs using large-scale surveys, namely a within-survey imputation exercise that derives predicted, otherwise unobserved, objective crop yields that stem from a machine learning model that is estimated with a random subsample of plots for which crop cutting and self-reported yields are both available. Using data from a methodological survey experiment and a nationally representative survey conducted in Mali, the analysis demonstrates that it is possible to obtain predicted objective sorghum yields with attenuated non-classical measurement error, resulting in a less biased assessment of the relationship between yields and agricultural inputs. The discussion expands on the implications of the findings for (i) future research on agricultural intensification, and (ii) the design of future surveys in which objective data collection could be limited to a subsample to save costs, with the intention to apply the suggested machine learning approach.

This paper is a product of the Development Data Group, Development Economics. It is part of a larger effort by the World Bank to provide open access to its research and make a contribution to development policy discussions around the world. Policy Research Working Papers are also posted on the Web at http://www.worldbank.org/prwp. The authors may be contacted at tkilic@worldbank.org or iyacouboudjima@worldbank.org.

_The Policy Research Working Paper Series disseminates the findings of work in progress to encourage the exchange of ideas about development issues. An objective of the series is to get the findings out quickly, even if the presentations are less than fully polished. The papers carry the names of the authors and should be cited accordingly. The findings, interpretations, and conclusions expressed in this paper are entirely those of the authors. They do not necessarily represent the views of the International Bank for Reconstruction and Development/World Bank and its affiliated organizations, or those of the Executive Directors of the World Bank or the governments they represent._ Produced by the Research Support Team
Survey Measurement Errors and the Assessment of the Relationship between Yields and Inputs in Smallholder Farming Systems: Evidence from Mali<sup>_∗_</sup> Isma ̈el Yacoubou Djima<sup>_†‡§_</sup> Talip Kilic<sup>_§_</sup> `ismael.yacouboudjima@psemail.eu tkilic@worldbank.org`

**JEL Codes:** C53, C83, Q12.

**Keywords:** Agricultural development; Smallholder farming; Agricultural inputs; Crop yields; Measurement error; Crop cutting; Machine learning; Household surveys; Mali; Sub-Saharan Africa.

> _∗_ The authors would like to thank (i) Karen Macours, Sylvie Lambert, Luc Behaghel, David Rhys Bernard, Sydney Gourlay, and the participants of the World Bank Living Standards Measurement Study (LSMS) Internal Seminar and the Casual Friday Development Seminar at the Paris School of Economics for their comments on the earlier drafts, (ii) ICRISAT-Mali for the successful implementation of the ERIVaS methodological survey experiment, and (iii) Berenger Djoumessi Tiague for his research assistance. We are grateful for the funding from the Mali Mission of the United States Agency for International Development (USAID) for the implementation of the LSMS–Integrated Surveys on Agriculture (LSMS-ISA)-supported surveys in Mali, including ERIVaS. This paper was produced with the financial support from the World Bank LSMS Program (worldbank.org/lsms) and the 50x2030 Initiative to Close the Agricultural Data Gap (50x2030.org), a multi-partner program that seeks to bridge the global agricultural data gap by transforming data systems in 50 countries in Africa, Asia, the Middle East and Latin America by 2030.

> _†_ Corresponding author.

> _‡_ PhD Candidate, Paris School of Economics.

> _§_ Living Standards Measurement Study (LSMS), Development Data Group, World Bank.
# **1 Introduction**

Small farms (with fewer than 2 ha) represent 84% of the estimated 570 million farms worldwide (Lowder et al., 2016). In view of the prevalence of smallholder farming across the developing world and the linkages between agricultural and welfare outcomes, one of the targets of the Sustainable Development Goal of Ending Hunger has been identified as doubling the agricultural productivity and incomes of small-scale food producers. One measure of agricultural productivity is crop yield, which is the amount of crop production harvested per unit of cultivated land. Increasing crop yields has been a long-standing goal of African governments and is key to feeding a growing global population while managing natural resources efficiently (Lobell et al., 2009; van Ittersum et al., 2013; Zhang et al., 2016).

Despite the seemingly simple formula for computing crop yields, and its frequent use in agricultural economics research, obtaining accurate measures of crop yields is underlined by a host of challenges in low- and middle-income countries, mainly due to reliance on household and farm surveys and measurement errors in self-reported information on crop production and cultivated area. Research has demonstrated the effects of these measurement errors on our understanding of the inverse-scale productivity relationship (IR) in agriculture – i.e. the finding that the productivity measures, including crop yields, are, on average, higher on smaller farms and plots vis-`a-vis their larger counterparts.

The first strand of research has focused on the denominator in the formula for crop yields: cultivated land area, and has documented systematic discrepancies between GPS-based and self-reported plot areas (Carletto et al., 2013; Carletto et al., 2015; Kilic et al., 2017). The cross-country comparable finding across these studies is that on average, farmers tend to over-report plot areas, and more so at the lower end of the plot area distribution. As a result of using GPS-based plot area measures, as opposed to their self-reported counterparts, research has shown that the IR, though weakened, remained statistically significant.

The second strand of research has focused on the numerator in the formula for crop yields: total crop production, which, in low- and middle-income countries, has traditionally been based on self-reported survey data. There are, however, several complications in eliciting accurate self-reported information on crop production, including (i) potential recall biases, in part as a function of when data are collected vis-`a-vis the harvest period (Wollburg et al., 2021), (ii) inclination to round off numbers (as shown by Gourlay et al. (2019) for crop production and by Carletto et al. (2015) for land area), and (iii) reliance on non standard measurement units, such as bunches and carts, and gaps in the availability and quality of conversion factors that are required to express non-standard measurement units in kilogram-equivalent terms and that are ideally differentiated for different conditions in which crop harvest may be reported by farmers (Oseni et al., 2017). Researchers have demonstrated that self-reported crop yields are subject to non-classical measurement errors, and are over-estimated on smaller plots, in comparison to objective measured yields based on crop cutting, i.e. the gold standard, albeit resource- and supervision-intensive, approach to crop yield estimation based on weighing the harvest of a randomly selected sub-section of a farmer’s plot. The key cross-country finding of interest is that at the plot-level the IR is strong and statistically significant when self-reported crop yields are used, and that the IR ceases to exist when crop cutting yields are used instead (Desiere and Jolliffe, 2018 and Abay et al., 2019 for Ethiopia; Gourlay 2
# et al., 2019 for Uganda).

Against this background, the first contribution of our paper is to provide empirical evidence from an alternative smallholder farming system in Mali on non-classical measurement errors (NCME) in self-reported (SR) crop yields vis-`a-vis their crop cutting (CC) counterparts on the same plots and demonstrate the consequences of NCME in SR yields for the estimated relationships between crop yields and agricultural inputs besides land, specifically seeds and labor. The literature’s heavy focus on land is understandable given that it is a primary factor of production. And our analysis fully corroborates the findings of Desiere and Jolliffe (2018), Gourlay et al. (2019), and Abay et al. (2019). However, with rising population densities and low levels of agricultural productivity, especially in Sub-Saharan Africa, more attention is being paid to the effects of measurement errors on the understanding of agricultural intensification (Abay et al., 2021) and of the extent of misallocation of inputs across farms (Gollin and Udry, 2021). In this context, understanding how crop yields relate to the use of other agricultural inputs is of great policy relevance.

To do so, however, it is necessary to ensure that the relationships are correctly assessed and are anchored in high quality survey data. Recognizing the technical and resource constraints that are faced by survey implementing agencies in contexts where data and yield improvements are needed the most, we present and validate an innovative approach to agricultural survey data collection and analysis in a way that requires the implementation of crop cutting only in a sub-sample of households/plots and that leverages machine learning techniques to derive predicted objective measures of crop yields for the sub-sample that is not subject to crop cutting. This constitutes the second contribution of our paper to the literature. In recent years, machine learning (ML) techniques have made their way into economics research and have proved their utility. Athey (2018), as part of an assessment of the early contributions of ML to economics, indicates that ML techniques have been most successful when applied to problems that require predictions as in our case. Furthermore, applying ML in agricultural economics research makes sense to model key variables such as crop yield, which is determined by a complex combination of soil quality, weather, input timing and other management choices, replete with non-linearities and interactions (Storm et al., 2020).

Our analysis showcases (a) the utility of ML techniques for predicting CC yields as a function of SR yields and complementary survey and geospatial data, and (b) the reliability of the resulting predicted CC yields in the analysis of the relationships between crop yields and agricultural inputs. We show analytically and empirically that predicted CC yields, although informed in part by SR yields, have attenuated NCME, which is uncorrelated with the intensity of agricultural input use. Only one-third of the sample is found to be sufficient for implementing crop cutting alongside self-reported survey data collection, such that an ML model can generate predicted CC yields for the rest of the sample in a way that leads to reliable parameter estimates of the relationships between yields and inputs. The findings reveal the importance of using SR yields as a predictor variable, and hence, underscores the need to continue to collect and improve the quality of self-reported survey data on crop production. Our analysis is first conducted with the data from a small-scale methodological survey experiment and is then replicated with the data from a nationally-representative survey to verify the external validity of 3
our conclusions. Beyond the methodological findings, our analysis has significant policy implications as it suggests that using self-reported yields, the returns of additional inputs per area of land are overstated and as such, it would indicate that agricultural intensification could have limited benefit in the context of the study.

The paper is organized as follows. Section 2 discusses the data and provides the descriptive statistics. Section 3 discusses how the choice of the yield measure affects the characterization of the relationship between land productivity and the agricultural inputs of interest, namely land area, seed quantity and household labor. Section 4 presents the machine learning framework and the results. Section 5 discusses the implications for survey data collection protocols. Section 6 concludes.

# **2 Data**

## **2.1 Context**

Our work is primarily informed by data collected during the 2017/2018 agricultural season as part of a methodological survey: the Enquˆete sur les Rendements et l’Identification des Vari ́et ́es du Sorgho (ERIVaS), focused on sorghum yield measurement and variety identification.<sup>1</sup> The fieldwork was conducted in Dio ̈ıla district, which is located in the region of Koulikoro, in south central Mali. Sorghum is a major crop in Mali, planted on 26% of plots cultivated by agricultural households, which account for 90% of rural households representing 80% of the population of Mali.<sup>2</sup> Koulikoro is one of the main growing regions of this staple.

ERIVaS was set up to assess the relative accuracy of self-reporting on land areas, sorghum varieties, and sorghum yields vis-`a-vis objective measurement methods, namely GPS for land area measurement, DNA fingerprinting for varietal identification, and crop cutting and satellite-based methods for crop yield estimation.<sup>3</sup>

## **2.2 Fieldwork details**

Four 10x10 km areas (henceforth referred to as strata) were geospatially delineated in Dio ̈ıla to ensure heterogeneity in terms of relief and terrain type. In each stratum, 150 households were to be selected and the number of households selected in each of the 17 villages located across the four strata was proportional to the total number of households in the villages found during the listing exercise. Given the low incidence of inter-cropping in our study area, only households with pure stand sorghum plots were considered for data collection. In each sampled household, one pure stand sorghum plot was randomly selected among the pure stand sorghum plots cultivated by the household. The result of this sampling exercise was 600 sorghum pure stand plots (one in each) of 600 households distributed in 17

> 1Sorghum yield and varieties identification methodological survey. The survey was designed as part of the World Bank LSMS methodological research agenda on measurement of crop yields in household surveys.

> 2Author’s calculation based on the Mali LSMS-ISA 2017 data.

> 3The analysis related to remote sensing measurement of yields and how the method compares to other methods is pursued in (Lobell et al., 2019). We compare part of the results obtained in both studies later in this analysis.

4
villages located in four strata across Dio ̈ıla. ERIVaS was conducted in three visits by semi-resident<sup>4</sup> enumerators.

During the first visit fieldwork, which was conducted in the post-planting period (SeptemberNovember 2017), each household received a light household questionnaire that elicited basic socioeconomic information, and this was augmented with an agriculture questionnaire that collected detailed information on farm organization, land tenure, crop cultivation and seed use, at either parcel- or parcelplot-level, depending on the topic. Following the random selection of the pure stand sorghum plot, the enumerator visited the plot location with the farmer, delineated the plot boundaries with the farmer and stored them on a handheld GPS unit, and set-up an 8x8m sub-plot for crop cutting, following the same protocol that was published by Gourlay et al. (2019) and that ensured the random placement of the sub-plot on each sampled plot. Each sub-plot was further divided into four 4x4m quadrants for which harvest was ultimately processed, weighed, and recorded separately.

During the second visit fieldwork, which was conducted immediately after the harvest (December 2017), the enumerators harvested the sub-plots with the help of the farmers and recorded the weight of the crop production immediately after the harvest and after additional drying at the enumerator’s residence. During the third visit fieldwork, which was implemented during the post-harvest period (February 2018), each household received an agriculture questionnaire that collected self-reported, parcel-plotlevel information on sorghum production and use of agricultural inputs, including fertilizer, household labor, and hired labor. Our analysis is based on 577<sup>5</sup> pure stand sorghum plots, with matching crop cutting and self-reported yield measures, augmented with plot- and household-level survey data and third-party geospatial data that are linked with georeferenced plot locations.

Intensive data quality controls and supervision measures were put in place to ensure that each crop cut sub-plot was placed in accordance with the desired protocol; was not managed by the farmer differently vis-`a-vis the rest of the plot; was not harvested by the farmer without the presence and supervision of the enumerator; and was harvested to ensure that production could be weighed and recorded separately for each quadrant, without including plants outside each quadrant.<sup>6</sup> These supervision measures contribute to the substantial additional costs associated with crop cutting and provide further impetus for our efforts to get more out of crop cutting, self-reported survey data and cutting-edge imputation techniques to improve agricultural survey data even when crop cutting may not be implemented at full scale. On the other hand, intensive supervision measures are unlikely to eliminate biases in SR yields<sup>7</sup> and these biases may in turn be correlated with self-reported information on the very agricultural inputs,

> 4One field team covered each stratum and resided in the most amenable village of that stratum throughout the

> 5Of the 600 plots selected, we drop 23 from this analysis for the following reasons: 18 crop cut subplots were harvested by farmers in the absence of the enumerators and as such had missing crop cut data, 3 plots had missing self-reported data due to a programming mistake, 1 plot had 100% self-reported loss but 0% enumerator assessed damage, and 1 plot was in the opposite situation: 100% enumerator assessed damage but 0% farmer reported loss.

> 6Appendix C.1 provides details of the fieldwork protocol followed for the crop-cut and the questionnaires fielded are available upon request.

> 7 When we compare household production reported by farmers on their selected plots with the non-selected pure sorghum plots, there is no statistically significant difference, indicating that they do not try to be more accurate when reporting the production on the selected plots.

5
whose relationships with crop yields we are trying to assess.

# **2.3 Descriptive statistics**

Table 1 presents summary statistics of the main plot and household level variables used in the analysis. The first part of the table displays plot characteristics. The variables extracted from the GPS coordinates show that most of the plots are located on similarly flat terrain within a 5 km radius of the household’s dwelling. In terms of land tenure, 95% of the plots are reportedly owned by the households cultivating them. The plot managers of these plots are mostly middle-aged men with low literacy levels. The average size of the associated households was 19,<sup>8</sup> and households were headed overwhelmingly by middle-aged men most of whom were illiterate. The average number of cultivated plots is further indication that agriculture is the main activity for the households. Furthermore, the average GPS-based plot area is considerably larger than the available estimates observed in smallholder farming systems elsewhere in Africa. For example, Desiere and Jolliffe (2018), Gourlay et al. (2019), Abay et al. (2019) report average plot sizes ranging from 0.12 to 0.37 hectares in their sample. The incidence of agricultural input use was, on average, 33% for hired labor and 24% for fertilizer.

Table 2 reports estimates for plot-level for sorghum production (kilograms) and yields (kilograms per hectare).<sup>9</sup> The competing figures are based on plot-level self-reported sorghum production versus estimated sorghum production that is tied to the crop harvest for the 8m x 8m crop cut sub-plot and separately, a randomly chosen 4m x 4m quadrant within the sub-plot.<sup>10</sup> All yield measures use the GPS-based plot areas. As we do not find statistically significant differences between crop cut yields as a function of the size of the crop cut, in the rest of the analysis, CC yields are tied to the 8m x 8m sub-plot.

We see that the average difference between SR yields and each of CC yield is statistically significant at the 1% level, and on average, SR yields are 21% higher than their CC counterparts. The distributional differences between SR and CC yields are, however, not statistically significant. On the other hand, although there are statistically significant distributional differences between SR and CC sorghum production estimates, but we find no statistical difference in the means. The fact that we find significant differences between means but not distributions for yield measures and vice versa for production measures can in part be explained by the standard errors, which are relatively larger at the mean in the case of the production measures, making it difficult to detect differences for the point estimates. These differences are picked-up when we conduct the tests of distributional differences.

> 8The household size may appear to be large but it is typical of rural households in the region: based on 153 observations in the Mali LSMS-ISA 2017 data, the mean number of individuals in households with at least one sorghum plot in the district of Dio ̈ıla is 17, with a 95% confidence interval of [14.9, 19.6].

> 9These results are robust to the treatment of outliers in the SR variable via winsorizing. See Table A1 in Appendix A.

> 10Total CC-based sorghum production is computed by multiplying the 8x8m CC yield with the GPS-based plot area. The harvest measure for one of the four 4m x 4m quadrants is randomly chosen in each of the 8m x 8m subplots to compute the yield and harvest for the CC 4m x 4m method.

6
**Table 1:** Summary Statistics of selected variables in ERIVaS sample

|Variable|Mean|Std.Dev|Min|Median|Max|Obs.|
|---|---|---|---|---|---|---|
|**Plot Characteristics**|||||||
|Slope (percent)|1.31|0.70|1.00|1.00|8.00|577|
|Elevation (m)|335.31|22.21|240.00|333.00|487.00|577|
|Potential Wetness Index|16.24|4.95|12.00|15.00|36.00|577|
|Plot has erosion problems<sup>_†_</sup><br>|0.20|0.40|0.00|0.00|1.00|577|
|Euclidean distance to dwelling (km)|2.11|2.01|0.00|1.66|18.39|577|
|Self-reported distance to market (km)<br>|7.55|5.40|0.00|7.00|30.00|577|
|HH owns the plot<sup>_†_</sup>|0.95|0.21|0.00|1.00|1.00|577|
|**Plot Manager Characteristics**|||||||
|Plot manager is female|0.01|0.08|0.00|0.00|1.00|577|
|Age of plot manager|52.81|14.24|15.00|53.00|97.00|577|
|Plot manager is literate|0.10|0.30|0.00|0.00|1.00|577|
|**Household Characteristics**|||||||
|HH size|19.14|11.79|2.00|16.00|60.00|577|
|Dependency ratio|1.15|0.53|0.00|1.12|3.50|577|
|Head of HH is female|0.01|0.08|0.00|0.00|1.00|577|
|Age of head of HH|55.96|14.52|18.00|57.00|97.00|577|
|Head of HH is literate|0.08|0.28|0.00|0.00|1.00|577|
|Number of plots cultivated by HH|5.26|2.33|1.00|5.00|18.00|577|
|Number of Sorghum plots cultivated by HH|1.50|0.81|1.00|1.00|7.00|577|
|Number of Cotton plots cultivated by HH|1.34|0.82|0.00|1.00|7.00|577|
|**Inputs Usage on plot**|||||||
|GPS measured area (ha)|1.56|1.25|0.05|1.20|12.02|577|
|Quantity of Seeds used (kg)|8.30|5.90|0.56|6.56|51.02|577|
|Household labor on plot (persons-days)|159.30|267.37|8.36|76.40|2711.45|577|
|Hired labor<sup>_†_</sup>|0.33|0.47|0.00|0.00|1.00|577|
|Hired labor (persons-days)|15.23|180.21|0.00|0.00|4195.02|577|
|Organic fertilizer<sup>_†_</sup>|0.24|0.43|0.00|0.00|1.00|577|
|Mineral/Chemical fertilizer<sup>_†_</sup>|0.30|0.46|0.00|0.00|1.00|577|
|Mineral/Chemical fertilizer (kg)|25.74|59.85|0.00|0.00|750.00|577|
|Pesticides<sup>_†_</sup>|0.50|0.50|0.00|1.00|1.00|577|
|**Yields (kg/ha)**|||||||
|Self-reported|636.9|822.3|0.0|400.0|8139.5|577|
|Crop-cut 8m x 8m|500.7|380.1|0.0|423.4|2472.5|577|
|Crop-cut 4m x 4m|501.0|402.4|0.0|419.4|2690.0|577|
|**Harvest (kg)**|||||||
|Self-reported (plots without crop-cut)|738.7|1000.2|0.0|438.5|10000.0|305|
|Self-reported|837.3|1147.3|0.0|488.7|9773.3|577|
|Crop-cut 8m x 8m|794.4|959.8|0.0|515.9|9253.2|577|
|Crop-cut 4m x 4m|782.2|980.0|0.0|463.3|9312.2|577|

_Notes:_ Dependent ratio is the ratio of number of members in the household less than 15 and more than 70 years old over the members between 15 and 70 years old. HH refers to household.<sup>_†_</sup> denotes a dummy variable.

7
**Table 2:** Comparison of plot-level measures of sorghum yields and harvests in ERIVaS

|**Y**|**ields (kg/ha)**|**Harvest (kg)**|
|---|---|---|
|_Means_|||
|SR|636.9|837.3|
|CC 8m x 8m|500.7|794.4|
|CC 4m x 4m|501.0|782.2|
|_Diference in Means_|||
|SR vs CC 8m x 8m|136.2***|42.9|
||(33.5)|(45.1)|
|SR vs CC 4m x 4m|135.9***|55.1|
||(33.6)|(47.0)|
|CC 8m x 8m vs CC 4m x 4m|-0.3|12.2|
||(6.1)|(11.0)|
|_Diference in Distribution_|||
|SR vs CC 8m x 8m||***|
|SR vs CC 4m x 4m|*|***|
|CC 8m x 8m vs CC 4m x 4m|||

_Notes:_ Standard Errors are shown in parentheses. Distributional differences are assessed using the Kolmogorov-Smirnov two-sample test. Statistical significance levels: *p _<_ 0.1; **p _<_ 0.05; ***p _<_ 0.01. Results with winsorizing of top 2.5 percent values of SR are shown in Table A1 in Appendix A

# **2.4 Correlates of measurement errors in self-reported vs. crop cut**

# **measure of yields**

Next, we showcase the drivers of the measurement errors in plot-level SR yields vis-`a-vis their CC counterparts. Given the myriad of covariates that can affect both yield measures, we use the ML algorithm LASSO to search through the pool of available covariates<sup>11</sup> and select the variables that are the most predictive of the difference: [log SR _−_ log CC]. Figure 1 shows the variables selected at least once after 100 simulations and their probability of inclusion. The key variables in this list include household labor, plot area, and quantity of seeds. These are the three inputs<sup>12</sup> that we focus on for the rest of the analysis.

To better get a sense of the direction in which the selected covariates impact measurement errors in yields, we regress [log SR _−_ log CC] on the selected variables. This is akin to how (Abay et al., 2019) and (Gourlay et al., 2019) explore the correlates of measurement errors in production data. While their choice of covariates is based upon the authors’ knowledge of the context and may appear arbitrary, our specification is more data driven and similar in spirit to an OLS-post LASSO estimation framework, which has the advantage of having a smaller bias even when the model is mis-specified (Belloni and Chernozhukov, 2013). The regression results are recorded in Table 3. The negative and statistically

> 11The covariates include factors of production that affect yields and field work related variables such as characteristics of respondents interacted with plot, households and enumerators dummies.

> 12It is striking that the dichotomous variables indicating the use of organic and chemical fertilizers do not appear as a key predictor of the difference of SR and CC yields. Per hectare use of chemical fertilizer may have carried a stronger signal but the quality of the data collected on this front was not satisfying so we did not further investigate the question for this input.

8
**Figure 1:** The graph shows variables with probability of inclusion greater than 0. 100 simulations are run on a training sample made of 80 percent of the ERIVaS sample. Labor, plot area and quantity of seeds are in log form. * indicates that the variable is interacted with enumerator fixed effect.

<!-- Start of picture text -->
*Respondent v1<br>age<br>Pre−harvest loss<br># of cotton plots<br>Rain fall<br>December<br>Rain fall<br>July<br>espondent literate<br>x # plots of Hh<br>Dammage<br> crop−cut<br>*Respondent v2<br>Manager<br>*Respondent v2<br>gender<br>*Respondent v2<br>age<br>*Respondent v1<br>Manager<br>*Respondent v1<br>gender<br>Unit SR harvest<br>Seeds<br>Plot area<br>Household labor<br>0.0 0.2 0.4 0.6 0.8<br><!-- End of picture text -->

9
significant coefficient on plot area indicates a tendency on the part of farmers to over-estimate their yields on smaller plots. The extent of over-estimation in SR yields is greater on plots with higher intensity of agricultural input use.<sup>13</sup> It is difficult to assert formally the mechanisms behind the correlates of the overestimation but knowing the context, it is possible to speculate on the potential explanations. For example, the number of cotton plots cultivated by the household is negatively associated with the farmers’ tendency to overestimate yields. With the ERIVaS study area being in the Malian cotton belt, farmers with cotton plots have received advice and assistance from agriculture extension agents. They are, therefore, likely to have better grasp of the production potential of their plots and to not overestimate their yields. There are additional findings that may be useful to keep in mind for future survey design. For instance, the extent of overestimation is reduced when the harvest is reported in large non-standard units. This is in part explained by the fact that plots with large harvests are of large size and are subject to underestimation of SR yields. This finding also suggests that allowing for non-standard measurement units in data collection and using conversion factors to express them in kilogram-equivalent terms reduce the discrepancies between SR and CC yields. Another correlate of yield overestimation is the enumerator assessed damage on the crop cut, indicating that farmers underestimate the effect on yields of the various damages that their plots may have been subject to during the growing season. More attention should be paid to the further piloting and potential inclusion of related questions in future surveys. Finally, plot managers (also more likely to be the respondents) who are literate have less tendency to overestimate their production, even with a higher number of plots cultivated, underscoring the need to work with literate respondents.

The analysis of the correlates of [log SR _−_ log CC] provides evidence that there is NCME in SR yields and NCME is negatively correlated with plot size and positively correlated with seed and labor inputs per hectare. From a policy standpoint, this implies that the returns to input intensification will be overestimated if SR yields are used instead of their CC counterparts. In other words, the systematic measurement errors in SR yields may be driving up the relationships between yields and inputs, and these relationships may in fact be artifacts of the data. We investigate these assertions analytically in the next section.

# **3 Assessing the relationship between yields and inputs**

## **3.1 Analytical framework**

This section attempts to assess how the errors in SR yields, in comparison to CC yields, affect the characterization of the relationships between land productivity and agricultural inputs, namely land area, seeds and household labor. We start out with the analytical framework proposed by Abay et al. (2019). The estimated yield input relationship is:

> 13This assumes that measurement errors in inputs are not correlated with SR too. We come back to this assumption later in the paper.

10
**Table 3:** Regression results of errors in yields against ML selected correlates of measurement errors in yields

||[logSR_−_<br>|logCC]<br>|
|---|---|---|
||OLS|IV|
|Household labor|0.191<sup>_∗∗∗_</sup><br>|0.173<sup>_∗∗∗_</sup><br>|
||(0.040)|(0.043)|
|Plot area|_−_0.259<sup>_∗∗∗_</sup><br>|_−_0.286<sup>_∗∗∗_</sup><br>|
||(0.092)|(0.096)|
|Seeds|0.148<sup>_∗_</sup>|0.148<sup>_∗_</sup>|
||(0.087)|(0.082)|
|Unit SR harvest|_−_0.121<sup>_∗∗∗_</sup>|_−_0.122<sup>_∗∗∗_</sup>|
||(0.038)|(0.038)|
|*Respondent v1 : female|1.433<sup>_∗∗∗_</sup>|1.410<sup>_∗∗∗_</sup>|
||(0.424)|(0.431)|
|*Respondent v1 : manager|0.415|0.433|
||(0.356)|(0.372)|
|*Respondent v2 : age|0.012<sup>_∗∗∗_</sup>|0.012<sup>_∗∗∗_</sup>|
||(0.004)|(0.004)|
|*Respondent v2 : female|0.0003|0.0004|
||(0.016)|(0.016)|
|*Respondent v2 : manager|_−_0.096|_−_0.095|
||(0.525)|(0.534)|
|Dammage crop-cut|0.169|0.159|
||(0.192)|(0.199)|
|Respondent literate x # plots of Hh|_−_0.063<sup>_∗∗∗_</sup>|_−_0.063<sup>_∗∗∗_</sup>|
||(0.024)|(0.024)|
|Rain fall - July|0.005|0.005|
||(0.004)|(0.004)|
|Rain fall - December|0.001|0.003|
||(0.029)|(0.029)|
|Number of cotton plots|_−_0.121<sup>_∗∗∗_</sup><br>(0.046)|_−_0.132<sup>_∗∗∗_</sup><br>(0.046)|
|Pre-harvest loss|_−_0.008<sup>_∗∗∗_</sup>|_−_0.008<sup>_∗∗∗_</sup>|
||(0.002)|(0.002)|
|*Respondent v1 : age|0.012|0.011|
||(0.015)|(0.015)|
|Enumerator Fixed efect?|Yes|Yes|
|Observations<br>|577|577|
|R<sup>2</sup>|0.301|0.302|
|Adjusted R<sup>2</sup>|0.256|0.256|

_Notes:_ Labor, plot area and quantity of seeds are in log form. In the second column, household labor is instrumentalized with household size. * indicates that the variable is interacted with enumerator fixed effect. v1 indicates postplanting visit and v2 indicates post-harvest visit. Enumerator fixed effect included and errors clustered by enumerator. Significance levels: *p _<_ 0.1; **p _<_ 0.05; ***p _<_ 0.01 11
_Y_<sup>_∗_</sup> _− A_<sup>_∗_</sup> = _β_<sup>_∗_</sup> ( _X_<sup>_∗_</sup> _− A_<sup>_∗_</sup> ) + _ε_ (1) where _ε_ is mean zero and uncorrelated with ( _X_<sup>_∗_</sup> _−A_<sup>_∗_</sup> ) and _β_<sup>_∗_</sup> can be estimated via ordinary least squares (OLS) regression. We consider the following type of non-classical measurement errors parameters:

- _δ_ : correlation of the measurement errors in the yield measure with true value of yield

- _λ_ : correlation of the measurement errors in the yield measure with the true value of input

- _α_ : correlation of the measurement errors in input with the true value of input

- _π_ : correlation of the measurement errors in yield with the measurement errors in the input measures

When present, _δ_ appears as a multiplicative attenuation bias. The remaining parameters enter in a general formula for an OLS estimate of _β_<sup>_∗_</sup> as following: where _ρx∗_<sup>2isthevarianceof</sup><sup>_X∗_and</sup><sup>_ρl_2thevarianceoftherandomcomponentofthemeasurement</sup> error in _X_ . The second and third term appear if and only if the input is present in the dependent and independent variables as in the case of plot area. To characterize the signs of the different parameters, we rely on the results of the regression presented in Table 3 and on Figure 2 which provides graphical evidence of the relationship between the difference of SR and CC yields – taken to be the measurement error in SR yields. If we assume that CC yields are the true value of yields, the difference between SR and CC is negatively correlated with the value of CC yields (see top left panel of Figure 2). This indicates that the sign of _δ_ in our data is negative. The case of the measurement error encapsulated in the parameter _δ_ is that of attenuation bias due to measurement error in the dependent variable.

Regarding _α_ , we note that the correlation of measurement error in our measure of plot area and true plot size is likely to be negligible because of the accuracy of GPS handheld devices (Carletto et al., 2017) and the large plot size in Mali. For labor, we can rely on the comparisons of recall- (as in our survey) versus diary-based approaches to survey data collection on labor, as reported by Arthi et al. (2018), Gaddis et al. (2020), who present evidence of overestimation in recall-based plot-level labor hours in Tanzania and Ghana, respectively, vis-`a-vis diary-based measures. As such, we expect a negative correlation between the measurement error in labor and its true value. Finally, characterizing _α_ is most complicated for quantity of seeds for which we do not have a benchmark for the true value, so we remain agnostic. Turning to _λ_ , for plot area, we use the GPS-based measure, which can be assumed reliably to have measurement errors that are negligibly correlated with the true measure. Top right panel of Figure 2 indicates that the relationship is negative, so we can take _λ_ to be negative for plot area. Characterizing the sign of _λ_ for labor and seed inputs is more difficult since they are based 12
**Figure 2:** Self-Reported yield overestimation of crop-cut yield over quintiles of crop-cut yields and key inputs. _Notes:_ The y-axis represents the amount by which SR yields overestimate CC yields ( _Y_ = _SR − CC_ ) in kg/ha. The x-axis shows the quintile of the crop-cut yields or of the input per ha.

**Crop−cut Yields**

**GPS Measured Area**

<!-- Start of picture text --> 1 2 3 4 5<br>300<br>100<br>0<br>−200<br><!-- End of picture text --> <!-- Start of picture text --> 1 2 3 4 5<br>500<br>300<br>100<br>−100<br><!-- End of picture text -->

**Quantity of Seeds per Ha**

**Days of Household Labor per Ha**

<!-- Start of picture text --> 1 2 3 4 5<br>600<br>400<br>200<br>0<br><!-- End of picture text --> <!-- Start of picture text --> 1 2 3 4 5<br>600<br>400<br>200<br>0<br><!-- End of picture text --> 13
on farmer-reporting and may be subject to measurement errors that are correlated with measurement errors in yield. However, let’s assume at first that this correlation is negligible. Having made this assumption, interpreting Figure 2 indicates that _λ_ is positive for these two inputs.

The bottom left panel of Figure 2 shows that in the first two quintiles of the quantity of seeds per hectare, SR yields are slightly underestimated vis-a-vis CC yields. In the third quintile, there is no statistically significant difference between the two yield measures. In the last two quintiles however, the over- estimation in SR yields is large and clearly statistically different from zero. A similar trend is observed for household labor per hectare (bottom right panel): As the number of days of household labor per hectare increases, there is a larger extent of over-estimation in SR yields. Table 3 includes additional evidence of the positive relationship between the errors in SR yields and the true values of household labor and quantity of seeds. We observe positive and statistically significant signs for labor and seeds, controlling for other drivers of measurement errors in SR yields.

Concerning labor, we note that the respondent literacy interacted with the number of plots - both contributors to the measurement error in recall-based plot-level labor inputs, per Arthi et al. (2018) and Gaddis et al. (2020) is among the covariates listed in Table 3. As such, it can be argued that the regression at least partially controls for measurement errors in household labor and that _λ_ , i.e. the correlation between the measurement error in yields and the true value of labor inputs, is positive. Using an instrumental variable approach, we can further augment the evidence base regarding the expected direction of _λ_ for labor. In the second column of Table 3, we regress the difference in SR and CC yields on household labor days using household size as an instrumental variable for household labor. Household size appears as a good candidate instrument for this exercise. It is directly correlated with household labor inputs, and is easier to measure, and thus, could enable us to partial out the measurement error in household labor. Additionally, the machine learning search for the drivers of measurement error in yields tells us that household size is not a significant contributor to this error. After using household size as an instrumental variable, the coefficient for labor remains positive and large. This finding provides further support to the assumption that measurement errors in yields is positively correlated with the true value of household labor.

Finally, we attempt to characterize _π_ the correlation of the measurement error of yields and the measurement error in inputs. In the case of plot area, considering that the measurement error in GPSbased plot area is negligible, the correlation with the measurement error in yield will also be negligible. For labor, Table 3 provides some sense of the sign of _π_ . We can see that the correlation between household labor and the difference between SR and CC yields is higher using a simple OLS versus an instrumental variable regression. Hence, there seems to be a positive relationship between measurement error in yields and measurement error in labor. At the same time, the coefficients across the two regressions are not statistically significantly different as such _π_ , as it is difficult to find an instrument that allows us to partial out the measurement errors in the quantity of seeds.

Table 4 provides a summary of the conjectures put forth regarding the parameters _δ_ , _λ_ , _α_ , and _π_ . For all inputs, _δ_ is negative so it it exerts a negative bias on _β_<sup>OLS</sup> but as noted by Abay et al. (2019), this is a multiplicative bias so it cannot reverse the sign of the relationship. The other parameters 14
however appear as additive terms. In the case of plot area, the most contributing parameter is _λ_ which is negative. As such, we expect a negative bias, as also documented in previous studies. For labor, _λ_ contributes positively to the bias, _π_ is positive and appears with a positive sign in the formula for _β_<sup>0LS</sup> ; and _α_ but will contribute positively to the bias since it will make the denominator of all terms in Equation 2 smaller, and, therefore, the multiplicative bias in the first term will go up (assuming _|α| <_ 1), and the bias from _λ_ and _π_ will grow larger. Hence, the overall bias in the coefficient _β_<sup>0LS</sup> for labor is expected to be positive. Finally, for quantity of seeds, we are able to only confidently sign _λ_ , which would contribute positively to the bias, but based on the descriptive evidence, we expect that the final bias will also be positive, similar to labor. We expect that this bias is essentially going to be driven by _λ_ i.e. the correlation of measurement error in yield with the true value of quantity of seeds. In sum, we expect that using SR yields in estimating the relationships between yields and inputs will result in underestimation of the returns to area and an overestimation of the returns to labor and quantity of seeds. The next section provides an empirical assessment of this assertion.

**Table 4:** Characterization of the NCME parameters in the assessment of yields - input relationship.

|Input|_δ_|_λ_|_α_|_π_|Bias|
|---|---|---|---|---|---|
|Area|_≤_0|_≤_0|_≃_0|_≃_0|Negative|
|Labor|_≤_0|_≥_0|_≤_0|_≥_0<br>most|likely positive|
|Quantity of seeds|_≤_0|_≥_0|-|-<br>most|likely positive|

# **3.2 Empirical results**

This section presents estimates of the yield - inputs relationship. We use simple bivariate regressions as presented in the earlier section. The equation for these regressions can be written as follows: We also estimate multivariate OLS regressions as is commonly done in the IR literature. Estimating a multivariate regression allows us to alleviate omitted variables bias concerns that could be raised in explaining the differences between the coefficients across the regressions of SR versus CC yields Desiere and Jolliffe (2018). The multivariate model is of the form: In both the bivariate and the multivariate equations above, _Y_ is the logarithmic transformation of the plot-level sorghum yield (kilograms per hectare), based on GPS-based plot area. _i_ denotes plot (recall that in ERIVaS, we select one plot per household), _A_ , _S_ , _L_ are respectively the logarithmic transformation of GPS-based plot area, the logarithmic transportation of the quantity of seeds used per 15
hectare, the logarithmic transformation of the number of days of household labor per hectare. _P_ is a vector of plot characteristics including inputs other than seeds and household labor, _O_ is a vector of objective measures of plot characteristics (including geospatial variables). _H_ is a vector of household characteristics, _M_ is a vector of plot manager characteristics. _V_ is a vector of dummy variables for villages (village fixed effects) and _εi_ the error term.

The coefficients of interest are the _β_ ’s. A negative and statistically significant _β_ 1 would be in support of the inverse scale-productivity relationship at the plot-level. A positive and significant _β_ 2 would indicate that increasing the quantity of seeds per hectare is correlated with higher yields, suggesting that one the potential way to increase yield is to increase the quantity of seeds per hectare. Finally a positive and significant _β_ 3 would indicate that more labor per hectare could potentially increase yields. From a policy perspective, a negative _β_ 1, and positive _β_ 2 and _β_ 3 plead in favor of agriculture intensification to improve yields.

Our empirical analysis confirms our expectations, as presented in the preceding section. Table 5 presents the regressions coefficients of key agricultural inputs, as obtained from the bivariate and multivariate regressions of SR and CC yields. The IR is always confirmed when using SR yields, with a statistically significant negative coefficient associated with plot area. More specifically, it appears that a 1 percent increase in GPS-based plot area results in a .2 percent reduction in sorghum yield. The relationship disappears, however, when using CC yields.

**Table 5:** Regressions results of SR and CC measures of yields for selected inputs

|||||Measu|res of Yi|elds|||
|---|---|---|---|---|---|---|---|---|
||SR|CC|SR|CC|SR|CC|SR|CC|
|GPS measured plot area|_−_0.268<sup>_∗∗_</sup>|0.138|||||_−_0.304<sup>_∗∗∗_</sup>|_−_0.045|
||(0.109)|(0.122)|||||(0.085)|(0.099)|
|Quantity of Seeds|||0.574<sup>_∗∗∗_</sup><br>(0.128)|0.051<br>(0.139)|||0.366<sup>_∗∗∗_</sup><br>(0.068)|0.185<sup>_∗∗_</sup><br>(0.086)|
|Days of household Labor|||||0.417<sup>_∗∗∗_</sup>|0.078<sup>_∗∗_</sup>|0.159<sup>_∗∗∗_</sup>|_−_0.040|
||||||(0.058)|(0.033)|(0.058)|(0.045)|
|Multivariate Framework?|No|No|No|No|No|No|Yes|Yes|
|Village Fixed efects?|No|No|No|No|No|No|Yes|Yes|
|Observations|577|577|577|577|577|577|577|577|
|R<sup>2</sup>|0.033|0.009|0.081|0.001|0.144|0.005|0.443|0.424|
|Adjusted R<sup>2</sup>|0.031|0.007|0.079|_−_0.001|0.143|0.003|0.391|0.371|

_Notes:_ Errors clustered at the enumeration area. Dependent and Independent variables shown here are log transformed. Full multivariate regression results are shown in Table A3 Significance levels: *p _<_ 0.1; **p _<_ 0.05; ***p _<_ 0.01 In sum, assuming that CC yields do not suffer from NCME, the errors in SR yields create spurious relationships between yields and key inputs in the context of smallholder farmers. The policy implications of these results are very concerning if agricultural initiatives are to be motivated by the analysis of these relationships. In fact, examining the full regression results in Table A3, it is striking how sensitive our estimations are to whether we use SR or CC yields. Using SR yields, plots with more intensive input use have higher yields, suggesting that the focus should be on agricultural intensification 16
and a broader access to or provision of seeds. Although the relation is still positive when using CC measure, the magnitude is considerably different - doubling the seeding rate would increase SR yields by 37 percent while it would only increase CC yields by 19 percent. For household labor inputs, the analysis based on SR yields points to the yield gains associated with increased labor inputs per hectare, suggesting a certain level of misallocation of labor across households. This suggestion, however, does not emerge if the analysis is anchored in CC yields.

These results emphasize the need for adopting objective measurement of production, specifically crop cutting, in household and farm surveys. The adoption of these methods would undoubtedly require additional technical and financial resources, and in the case of national surveys, could present considerable logistical challenges. In these circumstances, it is worth investigating whether crop cutting can be implemented in a sub-sample, with the intention to build an imputation model of CC yields as a function of easier-and-cheaper-to-collect self-reported and geospatial data, and to obtain, in the rest of the sample that is not subject to crop cutting, predicted CC yields that are subject to attenuated NCME and that allow for a less biased assessment of the relationships between yields and inputs. This is what is attempted in the second part of this analysis.

# **4 Estimating the relationships between yields and inputs using machine learning-based imputed crop cut yields**

This section provides an analytical framework on how the production function coefficients associated with agricultural inputs are expected to be modified as a result of using machine learning-based imputed crop cut yields (henceforth referred to as ML yields) vis-`a-vis CC and SR yields. This is followed by the presentation of the empirical results from the regressions of SR, CC and ML yields, verifying the predictions stemming from the analytical framework.

## **4.1 Analytical framework**

Recall that we are interested in estimating: Where _Y_<sup>_∗_</sup> is the log of production measure and _X_<sup>_∗_</sup> is the log of any explanatory variable of concern (labor, quantity of seeds used) and _A_<sup>_∗_</sup> is area. We do not observe the true value of _Y_<sup>_∗_</sup> nor _X_<sup>_∗_</sup> and _A_<sup>_∗_</sup> but we set aside measurement errors in _X_ and _A_ and focus on the measures of _Y_<sup>_∗_</sup> . We observe in the data _Y_<sup>_CC_</sup> , which is obtained from the enumerator weighted harvest on subplot crop cut and _Y_<sup>_SR_</sup> which is obtained from the farmer self-reported production. We can write both measures in terms of the true _Y_<sup>_∗_</sup> as : 17
Where _A_<sup>_∗CC_</sup> is the size of the crop cut subplot. In the case of _Y_<sup>_CC_</sup> , _u_<sup>_CC_</sup> is uncorrelated with _X_<sup>_∗_</sup> (the crop cut harvested weight is on a randomly chosen pre-delimited subplot, so there are no reasons to think that it will have measurement errors correlated with the measure of labor or quantity of seed, area). In that case, the measurement error in _Y_<sup>_CC_</sup> would not contribute to a bias in _β_<sup>_∗_</sup> .

Turning to production obtained through self-reporting _Y_<sup>_SR_</sup> , we documented earlier that _u_<sup>_SR_</sup> is subject to non-classical measurement error. Thus, we take _u_<sup>_SR_</sup> = _λ_ ( _X_<sup>_∗_</sup> _−A_<sup>_∗_</sup> )+ _ζ_ and expanding and simplifying the equation leads to _β_<sup>_OLS_</sup> = _β_<sup>_∗_</sup> + _λ_ .

We introduce _Y_<sup>_ML_</sup> _− A_<sup>_∗CC_</sup> which is yield predicted using a machine learning algorithm. For simplicity in derivation, we consider that _Y_<sup>_SR_</sup> _− A_<sup>_∗_</sup> is a linear term and express _Y_<sup>_ML_</sup> _− A_<sup>_∗CC_</sup> as: Where **W** is a vector of covariates that is used to predict yield not including _X_<sup>_∗_</sup> . So we end up with: Estimating Equation (1) using _Y_<sup>_ML_</sup> _− A_<sup>_∗CC_</sup> and expanding with Equation (5), we get that: By linearity, we can split the last line of the precedent equation in 4 terms (each on one line in the equation below): The first term simplifies to _γβ_<sup>_∗_</sup> . The second term can be written as _ηβ_<sup>_∗_</sup> since _F_ ( **W** ) is a predicted measure of _Y_<sup>_∗_</sup> uncorrelated with ( _X_<sup>_∗_</sup> _− A_<sup>_∗_</sup> ), we can assume that it will have some noise bringing an attenuation bias _η_ in the estimation of the OLS coefficient. The third term simplifies to _γλ_ . The last term Cov( _λζ_ + _w, X_<sup>_∗_</sup> _−A_<sup>_∗_</sup> ) goes to 0 since _λ_ is a constant, and _ζ_ assumed to be uncorrelated to _X_<sup>_∗_</sup> _−A_<sup>_∗_</sup> . Similarly, since **W** does not include input variables, _w_ is not correlated to _X_<sup>_∗_</sup> _− A_<sup>_∗_</sup> . So we end up with: 18
According to Equation (8), using a predicted measure of crop cut, we will be estimating _β_<sup>_∗_</sup> with an attenuation bias coming from the noise of the ML algorithm which will also contribute to attenuate the bias induced by the SR measure of yield.

# **4.2 Empirical application**

To test empirically whether it is judicious to use machine learning techniques to construct an alternate measure of crop yields based on a sub-sample of plot observations for which both crop cut and selfreported crop production data are available, we use the data from the ERIVaS 2017 experiment, which allows us to compare ML-predicted crop cut yields to their observed counterparts. Using data from a more controlled methodological study also ensures that the crop cut data are of the highest quality. A potential disadvantage, on the other hand, is that because the data are from a localized setting, it may be less heterogeneous, which could make it harder to accurately predict each observation. Our experimental protocol is as follows. First, we randomly select one-third of the observations in each ERIVaS village to construct a pooled data set to train the ML algorithm. The reason why we choose one-third of the observations to be part of the training data set is to mimic the extent of crop cutting conducted in the nationally-representative multi-topic household surveys that have been supported by the World Bank LSMS-ISA program. The remaining two-thirds of the data set serves as our test sample in which we compare the predicted yield measures obtained with ML against those based on farmer self-reporting and crop cutting, with the latter assumed to be closest to the “truth”. The statistics tied to this comparative assessment are computed over 100 simulations of training and test sample splits to ensure that are conclusions are not the result of one random realization, although in some parts of this subsection, we may present the results for one realization for better illustration purposes. Second, since our goal is to investigate whether the ML-based predicted crop yields are suitable for the analysis of the relationship between yields and key inputs, we estimate the relationships of interest in the test sample using the most “accurate” modeling set up and compare the resulting regression coefficients to the ones that are estimated with SR and CC yield measures.

## **4.2.1 Machine learning modeling**

We use an ensemble of methods<sup>14</sup> (LASSO, elastic-net, random forest, and gradient boosted trees)— a standard and popular machine learning technique— to generate yields predictions. We test the sensitivity of our estimation to the choice of the dependent variable (CC yields, CC harvest, in log or raw forms), and whether or not the predictors include SR yields. While the full technical details of the machine learning implementation are provided in section B of the Appendix, we discuss here the choice of the covariates considered for ML modeling.

On the whole, we fed the ML algorithms a set of 270 covariates across the following categories: (i) plot characteristics that exclude key plot inputs<sup>15</sup> but that include predictors related to land tenure, soil

> 14The technique is implemented using the R package SuperLearner (Polley et al., 2018).

> 15Since we will regress yields obtained using ML on key inputs, we do not include these variables in the set of ML covariates to ensure that the regression coefficients are not a mere mechanical result of the ML prediction model.

19
type, soil fertility measures, GPS-based Euclidian distance to the dwelling, (ii) plot-level agroclimatic and rainfall variables that are extracted based on georeferenced plot locations, (iii) plot-level agricultural practices, such as sowing dates and type of labor inputs, interacted with rainfall measures, (iv) household attributes, such as agricultural implement index, wealth index, and a series of household demographic attributes as well as socioeconomic characteristics of the head of household, and (v) other contextual information, including respondent attributes, day of visits to the households, and damage on crop cut sub-plots (as observed by the enumerators).

It is generally difficult to interpret the results of ML modeling, given the complexity of the algorithms, but we can get an insight on how the prediction of yields is made by examining the variables kept by the shrinkage algorithms or the importance of the random trees or forest algorithms. For illustration, Table 6 shows the result of the ML ensemble fit for the first sample split simulation when we fit the log CC and include log SR among the covariates. The fit that minimized the root mean squared error after cross-validation in the training sample was made with a weighted linear combination of a fit by LASSO, random forest and gradient boosting trees. The variables selected by LASSO are the log of SR and the enumerator assessed damage on the crop-cut. This makes sense as we expect SR yield to be a strong (although biased) proxy of CC yield. It is not surprising either that enumerator assessed damage on the crop cut appears as an important predictor since it allows to classify plots for which the harvest was completely lost. While these two variables are also important covariates for the random forest and gradient boosting trees algorithms, we also observe that farmer management practices, the interactions of input timing and amount of rainfall, and farmer access to agricultural equipment appear as top predictors of yields. Hence, the imputation approach allows the analyst to leverage the predictive power of SR yields, which has correlated measurement error, together with the predictive power of other determinants of yields, which suffer from less measurement error, to obtain an imputed measure of CC yields that would therefore have less NCME. We now turn to the assessment of the accuracy of the ML measure of yields.

# **4.2.2 Accuracy of the ML measure yields**

To understand which ML modeling set up gives the most accurate measure of yields, we compare the different ML predictions to CC yields in the test set using the adjusted _R_<sup>2</sup> and the coefficient of correlation ( _ρ_ ) between CC yields and the ML yields. To ensure that the assessment is not only the reflection of one random draw of the sample, we conduct this assessment with 100 different sample splits. The results are recorded in the top panel of Table 7. A couple of points emerge from examining this section of the table. First, predicting yields provides better results over predicting harvest. This makes sense since the dependent variable, CC is a method conceived to measure yields in the first place. Second, adding SR among the covariates considerably improves the accuracy of the ML predictions, indicating that it is a useful proxy of CC in our model. Third, it is better to estimate the model with the log-transformed dependent variable even when observed and predicted yields are compared in level terms. Therefore, we conclude that our best modeling set up is to fit the log-transformed CC yields including SR measure in the covariates.

20
**Table 6:** Machine Learning algorithm ensemble fit for the first sample split simulation. For illustration, we show the linear fit coefficients or predictive importance of selected variables for each ML algorithms chosen in the ensemble

||LASSO|Random forest|Gradient boosting trees|
|---|---|---|---|
|log SR yield|0_._190|35_._628|0_._132|
|Enumerator reported damage on the crop-cut|_−_0_._011|22_._164|0_._0001|
|Farmer reported percentage of crop loss on plot|0|9_._744|0_._0001|
|Agricultural implementation index|0|8_._963|0_._002|
|Village dummy|0|5_._701|0|
|Plot acquisition type|0|0_._414|0_._085|
|Sowing date 1st half June x rain fall 1st half June|0|0_._269|0_._131|
|Sowing date 1st half June x rain fall 2nd half May|0|0_._158|0_._132|
|Plot is owned by household|0|0_._103|0_._131|

_Notes:_ The ML model set up used log CC yield as the dependent variable and the covariates included log SR yield. The ML ensemble fit for the first sample split simulation comprised the predictions of LASSO (weight = 0.594), random forest (weight = 0.273), and gradient boosting trees (weight = 0.133). We show the variables selected (with non-zero coefficient) by LASSO and the top 5 most important variables in terms of predictive power for random forest and gradient boosting trees.

To put the accuracy of our ML measure in perspective in context, we also show the _R_<sup>2</sup> and _ρ_ for SR and crop cut-calibrated satellite yields, which we obtained from Lobell et al. (2019). These results are consigned in the bottom panel of Table 7. Our ML-based imputed CC yields, in comparison to SR yields, explains unambiguously more of the variation of in CC yields, and even more impressive finding is that the _R_<sup>2</sup> and _ρ_ of the are on-par with that of the CC-calibrated satellite yields.

# **4.2.3 Empirical assessment of the yields and inputs relationship using predicted yields**

We replicate bi-variate and multivariate regressions of yields as a function of key inputs, with competing measures of yields, including SR, CC and ML yields. We use our preferred model specification of ML in the first sample split simulation in the test sample. Since we are working with predicted measures, the standards errors for the regression coefficients are bootstrapped. Further, as a robustness check, we also use the method introduced by Chernozhukov et al. (2018) to compute de-biased regression coefficients and conservative standard errors when the number of control parameters is large as it is the case of the relationships of interest between yields and key inputs.

The regression results are reported in Table 8. We note that in the test sample, the observations that we previously made concerning the difference between the coefficients of the inputs when using SR or CC are still valid. For example, in the multivariate case, the quantity of seeds per hectare is positively correlated with yields but carry a stronger coefficient when the yield measure is SR, as opposed to CC. The regressions based on ML yields yield new findings. The coefficients obtained with ML yields are similar to the ones obtained with CC yields and the coefficient pairs are not statistically distinguishable, irrespective of the agricultural input in question. This is better seen in the bottom panels of Figure 21
**Table 7:** Performance of different ML modeling set-ups for predicting log yields in the test set over 100 different training - test sample splits.

||||log Yields|
|---|---|---|---|
|||_R_<sup>2</sup>|_ρ_|
|Dependent Variable<br>Covariat|es include SR|||
|log Yields|Yes|0.401|0.633|
|||[0.392, 0.409]|[0.627, 0.640]|
|log Yields|No|0.307|0.554|
|||[0.298, 0.316]|[0.546, 0.562]|
|log Harvest|Yes|0.242|0.492|
|||[0.233, 0.251]|[0.483, 0.501]|
|log Harvest|No|0.106|0.324|
|||[0.100, 0.113]|[0.313, 0.336]|
|SR||0.294|0.543|
|||[0.286, 0.301]|[0.535, 0.550]|
|Satellite Yields||0.135|0.370|
|||[0.131, 0.140]|[0.364, 0.376]|

_Notes: R_<sup>2</sup> is the adjusted _R_<sup>2</sup> of the regression of CC and the corresponding measure of yields. _ρ_ is the correlation between CC and the measure of corresponding measure of yields. We report the average _R_<sup>2</sup> and _ρ_ in the test set (N = 386) of 100 test sample splits and the 95% confidence interval in brackets.

3 in which we plot the coefficients for area, quantity of seeds per hectare, and labor per hectare with their 95 percent confidence interval, as obtained from the regressions of SR, CC and ML yields. It is apparent from the figure that the coefficients from the regressions of CC and ML yields are statistically equivalent. The coefficients obtained using the method put forth in (Chernozhukov et al., 2018) also provide estimates that are equivalent to the bootstrapped ones (see bottom left panel in magenta).

Furthermore, repeating the estimation of the multivariate yield-inputs regression for the 100 sample split simulations, we find that for all the simulations, the ML yield regression coefficient for plot area is within the 95 percent confidence interval of the comparable coefficient from the CC yield regression. For quantity of seeds per hectare and labor inputs per hectare, the coefficients estimated with ML measure of yield are within the 95 percent confidence interval of the comparable coefficients from the CC yield regression, in 96 and 80 of the simulations, respectively, out of 100. These results are illustrated on the left panel of Figure 4. The red and green kernel density plots represent the distribution of the coefficients when the relationship is estimated with SR and CC yields, respectively, and the blue histogram bars represent the distribution of the comparable coefficient when the relationship is estimated with ML yields. The middle panel of plots show the results of the regressions of ML yields that are obtained without including SR yields among the covariates. Compared to the aforementioned results based on ML yields, the ML yield regression coefficient for labor input per hectare is in fact more frequently within the 95 percent confidence interval of the comparable coefficient from the CC yield regression (compare bottom left and bottom middle density plots).

Under the assumption that the correlation of measurement error of labor with itself ( _α_ ) and with the measurement error of CC yield ( _π_ ) are both negligible and that we capture an unbiased coefficient 22
|ML_dml_|_−_0.002<br> (0.061)<br>0.038|(0.079)<br>0.052|(0.045)|Yes|386|ard<br>were<br> 100|
|---|---|---|---|---|---|---|
|ML_b_|_−_0.069 <br> (0.056) <br>0.106_∗∗_|(0.050) <br>0.049|(0.035)|Yes|386|the stand<br>rd errors <br>d conduct|
||<br>|||||nd<br>a<br>n|
|CC|0.041<br>(0.128)<br>0.221_∗∗_|(0.098)<br>0.013|(0.039)|Yes|386|ession a<br>he stand<br>le in 5 a|
|SR|_−_0.211_∗∗_<br>(0.105)<br>0.372_∗∗∗_|(0.065)<br>0.246_∗∗∗_|(0.068)|Yes|386|ia OLS regr<br>cients and t<br>lit the samp|
|ML_b_||0.131_∗∗∗_|(0.039)|No|386|computed v<br>ession coef<br> LASSO, sp|
|ields<br>CC||0.095_∗∗_|(0.045)|No|386|ients were <br>e ML regr<br>on, we use|
|sures of Y<br>SR||0.448_∗∗∗_|(0.060)|No|386|beta coefc<br>cates that th<br>his estimati|
|Mea<br>ML_b_|0.106|(0.088)||No|386|s that the <br>L_dml_ indi<br>hod. For t|
|CC|0.075|(0.157)||No|386|indicate<br>tions. M<br>ning met|
|SR|0.582_∗∗∗_|(0.136)<br>||No|386|nd CC. ML_b_<br>h 100 replica<br>achine lear<br>1|
|ML_b_|_−_0.023<br> (0.074)|||No|386|l for SR a<br>strap wit<br>) double m<br> ***p_<_0.0|
|||||||ve<br>ot<br>8<br>;|
|CC|0.139<br>(0.107)|||No|386|illage le<br>d via bo<br> al. (201<br>**p_<_0.05|
|SR|_−_0.302_∗∗_<br>(0.119)<br>s per ha|bor per ha||efect?<br>No|386|d errors clustered at the v<br>L predictions are obtaine<br>wing the Chernozhukov et<br>nifcance levels: *p_<_0.1;|
||eed|la||ed|s|dar<br>e M<br>llo<br> Sig|
||ea<br>ty S|f Hh||Fix|ation|Stan<br>or th<br>ted fo<br>ions.|
||r<br>ti|o||e|v|_:_ <br>s f<br>u<br>it|
||Plot a<br>Quan|Days||Villag|Obser|_Notes_<br>error<br>comp<br>repet|

23
**Figure 3:** Distributions and yield-input regression coefficients for different measures of yield in the first sample split of the test set N = 386. _Notes:_ Left top panel shows the kernel density of the different measures of yield. The right middle panel shows the scatter plots of CC vs ML yields. The right middle panel shows the scatter plots of CC vs SR yields. The bottom left/right panel shows the bivariate/multivariate yield-input regression coefficients for different measures of yield.

<!-- Start of picture text --> G CC<br>ML<br>SR<br>0 500 1000 1500 2000 2500<br>Yield (kg/ha)<br>0.5<br>G<br>G G<br>0.0<br>−0.5<br>Area Seeds HH Labor<br>0.0030<br>0.0025<br>0.0020<br>Density 0.0015<br>0.0010<br>0.0005<br>0.0000<br>β<br><!-- End of picture text --> <!-- Start of picture text --> 0 500 1000 1500 2000 2500<br>SR Yield (kg/ha)<br>0 200 400 600 800<br>ML Yield (kg/ha)<br>0.4<br>0.2 G<br>G<br>0.0 G<br>−0.2<br>−0.4<br>Area Seeds HH Labor<br>2500<br>1500<br>CC Yield (kg/ha)<br>500<br>0<br>2000<br>1000<br>CC Yield (kg/ha)<br>500<br>0<br>β<br><!-- End of picture text --> 24
**Figure 4:** Distribution of yields-inputs regression coefficients in the test for 100 different sample splits. The red and green kernel density plots represent the distribution of the coefficients when the relationship is estimated with SR and CC yields, respectively, and the blue histogram bars represent the distribution of the comparable coefficient when the relationship is estimated with ML yields. The left panel shows the distribution when in the machine learning model, we include SR. The middle panel of plots show the results of the regressions of ML yields that are obtained without including SR yields among the covariates. The right panel shows the results from the alternative regressions of ML yields which are computed with ML-based imputed measures of CC harvest that are in turn standardized by plot area (as opposed to imputing ML yields directly)

<!-- Start of picture text --> GPS measured plot area GPS measured plot area<br>−0.8 −0.4 0.0 −0.8 −0.4 0.0<br>15 20<br>10<br>10<br>5<br>5<br>0 0<br><!-- End of picture text --> <!-- Start of picture text --> GPS measured plot area<br>−0.8 −0.4 0.0<br>8<br>6<br>4<br>2<br>0<br><!-- End of picture text --> <!-- Start of picture text --> Quantity of Seeds Quantity of Seeds<br>0.0 0.2 0.4 0.0 0.2 0.4<br>10<br>12<br>8<br>6 8<br>6<br>4<br>4<br>2 2<br>0 0<br><!-- End of picture text -->

**Quantity of Seeds**

<!-- Start of picture text --> 0.0 0.2 0.4<br>8<br>6<br>4<br>2<br>0<br><!-- End of picture text -->

**Days of household Labor**

**Days of household Labor**

**Days of household Labor**

<!-- Start of picture text --> −0.1 0.0 0.1 0.2 −0.1 0.0 0.1 0.2 −0.1 0.0 0.1 0.2<br>β β β<br>12<br>20 20<br>8<br>10 10 6<br>4<br>5 5<br>2<br>0 0 0<br><!-- End of picture text --> 25
for labor with the use of CC yields, these results suggest that if we suspect a high degree of correlation between the measurement error of labor with the true value of labor, it may be better to estimate the yield-input relationship using a predicted measure of CC yields that does not include SR yields among the list of predictors. On the other hand, if _λ_ is not high (as it seems to be the case with quantity of seeds per hectare), it is better to include SR yields among the list of predictors for CC yields. In any case, it is always better to use the ML yields rather than SR yields. Finally, the right panel of Figure 4 shows the results from the alternative regressions of ML yields which are computed with ML-based imputed measures of CC harvest that are in turn standardized by plot area (as opposed to imputing ML yields directly). We can see that the coefficients associated with the key inputs are not estimated correctly - especially for plot area. This confirms that deriving ML-based imputed CC yields, as opposed to CC harvests, is the most appropriate.

To further confirm that the use of ML yields de-biases the estimation of the relationships between yields and inputs, we estimate regression, as presented in Table 3, using the test sample and both SR and ML yields. The results are presented in Table 9. First, inputs do not correlate with overestimation of ML(SR,W) as much as they do with SR. In fact, none of the coefficients are significant. And although the coefficient for quantity of seeds per hectare was positive but not significant under the use of SR yields in the test sample, it is now reversed and is no longer significant. Second, the adjusted _R_<sup>2</sup> decreases considerably which indicates that the covariates are associated to a lesser extent with the difference in values between CC and ML yields.

# **4.2.4 Application on the nationally representative survey**

To assess the external validity of our findings thus far, we replicate the analysis with data from the Enquˆete de Conjoncture Int ́egr ́ee (EACI 2017)<sup>16</sup> a nationally-representative multi-topic household survey, spanning both rural and urban areas of Mali and with detail questionnaire modules on smallholder agricultural activities. The households were interviewed by semi-resident enumerators in three visits, following a fieldwork protocol that was similar to that of ERIVaS. All major crops, including sorghum, were subject to crop cutting during the EACI fieldwork, and the size of the randomly-placed crop cut sub-plot on sorghum plots specifically was 5m x 5m (opposed to 8m x 8m in ERIVaS). One of the objectives of EACI is to derive national- and regional-level estimates of total crop production and yields. On yield measurement, plots, rather than households, are the primarily units of analysis, with direct implications on the sampling of plots for crop cutting purposes. Specifically, in each sampled EACI enumeration area, a complete listing of plots was conducted, and the listed plots were organized into separate subsets in accordance with the cultivated crop (or crop combinations, in the case of intercropped plots). A third of plots were then selected at random from each subset. This means that households with more plots had a greater probability of having a plot selected for crop cutting. In the end, we have farmer self-reported production data for about 24,000 plots of which 7,800 were subject

> 16The EACI 2017 is also the Mali LSMS-ISA 2017. It was built upon the Agricultural Conjuncture Survey which is implemented yearly by the Statistical Unit of the Ministry of Agriculture. For the 2017 edition, living conditions modules were added on a subsample as part of the Mali LSMS-ISA project.

26
**Table 9:** Regression results of errors in yields against correlates of measurement errors in yields

||Measurement|Errors in Yields|
|---|---|---|
||log SR- log CC|log ML - log CC|
|Household labor|0.210<sup>_∗∗∗_</sup>|0.028|
||(0.042)|(0.061)|
|Plot area|_−_0.283<sup>_∗∗∗_</sup>|_−_0.169|
||(0.106)|(0.104)|
|Seeds|0.087|_−_0.123|
||(0.105)|(0.131)|
|Unit SR harvest|_−_0.076<sup>_∗∗_</sup>|_−_0.003|
||(0.034)|(0.035)|
|*Respondent v1 : female|1.733<sup>_∗∗∗_</sup>|1.903<sup>_∗∗∗_</sup>|
||(0.455)|(0.381)|
|*Respondent v1 : manager|0.519|0.376|
||(0.359)|(0.430)|
|*Respondent v2 : age|0.015<sup>_∗∗∗_</sup>|0.010<sup>_∗∗_</sup>|
||(0.005)|(0.005)|
|*Respondent v2 : female|_−_0.017|_−_0.0005|
||(0.016)|(0.015)|
|*Respondent v2 : manager|_−_0.331|_−_1.266<sup>_∗∗∗_</sup>|
||(0.449)|(0.409)|
|Dammage crop-cut|0.160|_−_0.567<sup>_∗∗_</sup>|
||(0.175)|(0.242)|
|Respondent literate x # plots of Hh|_−_0.083<sup>_∗∗∗_</sup>|_−_0.031<sup>_∗_</sup>|
||(0.018)|(0.016)|
|Rain fall - July|0.003|0.004|
||(0.003)|(0.004)|
|Rain fall - December|0.046|_−_0.005|
||(0.033)|(0.037)|
|Number of cotton plots|_−_0.061|_−_0.058|
||(0.069)|(0.044)|
|Pre-harvest loss|_−_0.010<sup>_∗∗∗_</sup>|_−_0.0004|
||(0.003)|(0.002)|
|*Respondent v1 : age|_−_0.008|_−_0.007|
||(0.016)|(0.015)|
|Observations|386|386|
|R<sup>2</sup>|0.361|0.150|
|Adjusted R<sup>2</sup>|0.297|0.065|

_Notes:_ Estimation conducted in the test sample of the first sample split simulation. Labor, plot area and quantity of seeds are in log form. * indicates that the variable is interacted with enumerator fixed effect. v1 indicates post-planting visit and v2 indicates post-harvest visit. Enumerator fixed effect included and errors clustered by village. Significance levels: *p _<_ 0.1; **p _<_ 0.05; ***p _<_ 0.01 27
to crop cutting. Focusing on sorghum, EACI collected data on 3,569 plots of which 1,098 were subject to crop cutting.

We start this part of the analysis by comparing the data from ERIVaS and EACI. This provides clues regarding the potential modifications, if any, in the parameter estimates for the relationships between yields and inputs when we extend our analysis to a nationally-representative survey. Table 10 shows summary statistics for pure stand sorghum plots on in ERIVaS and EACI samples.<sup>17</sup> We see that the measures of yields and inputs are statistically different between the two surveys. The mean value in EACI is often larger, and as can be observed on Figure A2 in the Appendix, the amplitude is often greater for the inputs. The latter finding reflects the fact that EACI covers the entire country and captures more heterogenous farming practices and outcomes. This may in turn be beneficial for the use of ML techniques to derive imputed CC yields, since these algorithms work best with more observations and are more adapted to handle potential heterogeneity across units of observations.

**Table 10:** Sample summary statistics of ERIVaS and EACI 2017.

|Variable|Sample|Mean|Std.Dev|Min|Median|Max|Obs.|
|---|---|---|---|---|---|---|---|
|SR Yields|ERIVaS|636.86|822.26|0.00|400.00|8139.45|577|
||EACI 2017|536.70<br>***|874.18|0.00|317.89|9523.81|3237|
|CC Yields|ERIVaS|500.68|380.06|0.00|423.44|2472.50|577|
||EACI 2017|760.19<br>***|482.30|0.00|720.00|3200.00|980|
|GPS measured area (ha)|ERIVaS|1.56|1.25|0.05|1.20|12.02|577|
||EACI 2017|2.65<br>***|2.49|0.02|1.94|17.83|3237|
|Quantity of Seeds used (kg/ha)|ERIVaS|8.30|5.90|0.56|6.56|51.02|577|
||EACI 2017|13.38<br>***|42.57|0.00|5.57|1300.00|3237|
|Household labor on plot (persons-days/ha)|ERIVaS|159.30|267.37|8.36|76.40|2711.45|577|
||EACI 2017|69.03<br>***|146.20|0.54|34.04|2700.00|3237|

_Notes:_ To ensure comparability of the variables, the EACI 2017 sample is limited to pure stand plots. The stars. below the sample mean values of the two surveys denote the statistical significance level for a t-test comparison of the means in the two samples. Significance levels: *p _<_ 0.1; **p _<_ 0.05; ***p _<_ 0.01 In terms of the non-classical measurement errors that were uncovered in the ERIVaS data, similar patterns emerge from the EACI sample. Table 11 shows the means of alternative yield and harvest measures for pure stand and intercropped sorghum plots. Similar to ERIVaS, the mean differences between SR and CC measures are statistically significant. However, in the nationally-representative sample, the sign of the difference is opposite and we find that the mean SR yields are underestimated vis-`a-vis CC yields (i.e., SR - CC _<_ 0). Figure 5 allows us to better understand why this is the case by showcasing the mean difference between SR and CC yields across quintiles of plot area, quantity of seeds per hectare and days of household labor per hectare. SR yields are overestimated vis-`a-vis CC yields in the first plot area quintile, while, in each of the remaining quintiles, SR yields are underestimated. On

- 17An expanded set of descriptive statistics computed with both samples can be found in Table A4.

28
the other hand, SR yields are underestimated vis-`a-vis CC yields in each of the first four quintiles of quantity of seeds per hectare and days of household labor per hectare and are conversely overestimated in the top quintile.

**Table 11:** Comparison of plot-level measures of sorghum yield and harvest in EACI 2017

||**Y**<br>All|**ields (Kg/**<br>Pure-stand|**Ha)**<br> Intercropped|All|**Harvest (K**<br>Pure-stand|**g)**<br> Intercropped|
|---|---|---|---|---|---|---|
|_Means_|||||||
|SR|574.6|561.1|687.1|970.6|950.8|1135.1|
||(27.2)|(28.0)|(101.0)|(43.6)|(44.1)|(174.3)|
|CC|757.0|760.2|730.5|2037.1|2052.9|1906.5|
||(14.5)|(15.4)|(42.5)|(85.9)|(91.9)|(237.6)|
|_Diference in Means_|||||||
|SR vs CC|-182.4***|-199.1***|-43.4|-1066.5***|-1102.0***|-771.4***|
||(29.1)|(30.3)|(99.5)|(77.9)|(84.1)|(193.9)|
|_Diference in Distribution_<br>SR vs CC|***|***|***|***|***|***|

_Notes:_ Distributional differences are assessed using the Kolmogorov-Smirnov two-sample test Significance levels: *p _<_ 0.1; **p _<_ 0.05; ***p _<_ 0.01 The differences in the graphs based on ERIVaS versus EACI samples are due, in part at least, to the differences in the distributions of the input variables. For example, the mean SR-CC difference in the third quintile of plot area in EACI is equivalent to the mean difference in the fourth quintile in ERIVaS, so the underestimation trend that we observe in ERIVaS starts earlier in the distribution for EACI 2017. However, the direction of the correlation between the SR-CC difference and the quantity of inputs per hectare are the same in both ERIVaS and EACI samples. As such, it is not surprising to see in Table 12 that the coefficients of the bivariate and multivariate regressions of yields and inputs change significantly based on whether we use SR versus CC yields in the EACI data, but that the changes are fully consistent with the findings based on the data from the methodological experiment.

Furthermore, while we do not have concerns regarding measurement errors in CC yields in the ERIVaS data set due to (i) strict supervision of activities related to crop cut sub-plot placement, harvesting and weighing, and (ii) comprehensive quality control measures applied to the incoming data throughout the fieldwork, there is reason to believe that the EACI crop cutting operations were not implemented as successfully. Figure 6 shows a scatterplot of SR and CC yields in the EACI data set and there is evidence of rounding errors in CC yields. Assuming that these errors are not correlated with the measures of inputs,<sup>18</sup> this would mostly create an attenuation bias in the coefficients for the respective inputs in the regressions of CC yields and could, therefore, explain in part the zero coefficients in Table 12. This finding underscores the need of quality (but still scalable) digital measurement devices when objective measurement is the goal.<sup>19</sup>

> 18We created a variable indicating whether an observation was rounded and it appears not to be associated with the value of the key inputs we study here. A close look at this issue is left to future work.

> 19In fact, discussion with CPS/SDR confirmed that during this round, the survey enumerators relied on scales which lacked accuracy and were scheduled to be replaced.

29
**Figure 5:** Self-Reported yield overestimation of crop-cut yield over quintiles of crop-cut yields and key inputs. _Notes:_ The y-axis represents the amount by which SR yields overestimate CC yields ( _Y_ = _SR − CC_ ) in kg/ha. The x-axis shows the quintile of the crop-cut yields or of the input per ha. We are restricted to the plots with crop-cut

**Crop−cut Yields**

<!-- Start of picture text --> GPS Measured Area<br><!-- End of picture text --> <!-- Start of picture text --> 1 2 3 4 5<br>200<br>0<br>−400<br>−800<br><!-- End of picture text --> <!-- Start of picture text --> 1 2 3 4 5<br>600<br>200<br>0<br>−400<br><!-- End of picture text -->

**Quantity of Seeds per Ha**

**Days of Household Labor per Ha**

<!-- Start of picture text --> 1 2 3 4 5<br>600<br>200<br>0<br>−400<br><!-- End of picture text --> <!-- Start of picture text --> 1 2 3 4 5<br>600<br>200<br>0<br>−400<br><!-- End of picture text --> 30
<!-- Start of picture text --> a aa<br>_<br>o o a . 7<br>oo a ‘ a” 7<br>7<br>QB Oo 7” ”<br>7<br>.<br>o oo oo o “o O<br>oo ge " 0 9 nD g o° a “<br>o Oo o 2 oO .”<br>& om oa , 9 o o<br>Do 4<br>poe m om FF a7<br>oO oO o 7,%<br>ao mp Accs | a o :<br>=a Bam til=a eh OD, ¢ a a<br>copie? tt So cf oo . " o o<br>Pilea ieeeeect ae Reet i a)v2 aluyepecOOo 7 ca) BB 4 o- o oO oo 5 “ S<br>i °<br>Hommes OC ae oa Qo a<br>: Ae ern) Sau Peete eccrin ae iL)t e t‘4CT)8Oo"0Cee e 8 oa oO oao0 i) 5 a a a ’ :<br>4<br><!-- End of picture text -->
Nevertheless, we replicate the ML approach, as outlined in section 4.2.1, with the nationallyrepresentative EACI sample. Figure 7 summarizes the results for the first sample split out of the 100 replications. The right top-panel shows that ML yields 40 percent of the variation in CC yields, and the level of explained variation is in fact higher than the comparable result based on the ERIVaS sample. Even though we continue to see (in the left-top panel) a higher concentration of values around the mean for ML yields, there is a better tracking of the large values of CC yields. The coefficients for the key inputs in the regressions of ML yields are also comparable to those obtained from the regressions of CC yields, as shown in the bottom two plots of Figure 7. While the size and heterogeneity of the EACI sample are among the factors that improve the predictive accuracy of the ML model vis-`a-vis the results achieved with the ERIVaS sample, the EACI rounding errors in CC yields may also be creating a scenario that ML tools are better equipped to accommodate.

To verify whether we can expect the same results from the ML approach done with ERIVaS (selecting randomly a third of the plots) using the EACI 2017 and make inferences to the population of sorghum plots in Mali, we compare in Table A5, plots and households characteristics of plots selected for crop cutting with those that were not. The statistically significant differences appear to be seldom providing confidence that the selection of plot for CC is analogous to the simulation conducted with the data from the experience.

Finally, Table 12 reports the results from the yield regressions estimated with ML, and separately SR, yields using the entire EACI sample, and the results from the yield regressions estimated with CC yields using the sub-sample of EACI plots that were subject to crop cutting. The coefficient for plot area is negative but not statistically significant. This may be because the mean difference in plot area between the crop cut plots and the rest of the sample was statistically significant. The plots that were not subject to crop cutting were, on average, larger, and in view of the evidence regarding the underestimation in SR yields on larger plots, the IR is weakened when the regression is estimated using the entire sample. And while the coefficients for key inputs are consistently positive and statistically significant in the regressions of SR yields, the comparable coefficients in the regressions of ML yields are not statistically significant and are consistent with those estimated in the regressions of CC yields. These results are fully consistent with the findings based on the ERIVaS data set and raise flags concerning the analyses conducted with SR yields.

# **5 Implications for crop yield measurement in household and farm surveys**

Having demonstrated the promise of the use of ML to derive imputed measures of CC yields, this section provides a few recommendations on what data to collect and how to collect it in an efficient way.

First, given the importance of using SR yields to improve the quality of the ML-based imputed CC yield predictions, survey practitioners should continue to collect farmer-reported information on crop production, and reduce NCME in the resulting data, including by reducing length of farmer recall and improving the completeness and quality of conversion factors for non-standard measurement units, 32
**Figure 7:** Distributions and yield-input regression coefficients for different measures of yield in one sample split N = 732. _Notes:_ Left top panel shows the kernel density of the different measures of yield. The right middle panel shows the scatter plots of CC vs ML yields. The bottom left/right panel shows the bivariate/multivariate yield-input regression coefficients for different measures of yield.

<!-- Start of picture text --> G CC<br>ML<br>SR<br>0 1000 2000 3000<br>Yield (kg/ha)<br>0.6<br>0.4<br>0.2<br>G<br>G<br>0.0<br>G<br>−0.2<br>−0.4<br>Area Seeds HH Labor<br>0.0015<br>0.0010<br>Density<br>0.0005<br>0.0000<br>β<br><!-- End of picture text --> <!-- Start of picture text --> 0 500 1500 2500<br>ML Yield (kg/ha)<br>0.8<br>0.6<br>0.4<br>0.2<br>0.0 G G G<br>−0.2<br>Area Seeds HH Labor<br>2500<br>1500<br>CC Yield (kg/ha)<br>500<br>0<br>β<br><!-- End of picture text --> 33
||ML_b_|0.089|(0.063)<br>0.012|(0.033)<br>_−_0.038|(0.056)|Yes|3,569|ession and||
|---|---|---|---|---|---|---|---|---|---|
||CC|0.003|(0.050)<br>0.055_∗_|(0.031)<br>_−_0.050|(0.056)|Yes|1,098|OLS regr|1|
||SR|_−_0.173_∗∗∗_|(0.053)<br>0.151_∗∗∗_|(0.035)<br>0.143_∗∗_|(0.058)|Yes|3,569|omputed via|.05; ***p_<_0.0|
||ML_b_|||_−_0.112_∗_|(0.059)|No|3,569|ents were c|0.1; **p_<_0|
||CC|||0.006|(0.045)|No|1,098|ta coefci|evels: *p_<_|
|of Yields|SR|||0.396_∗∗∗_|(0.096)|No|3,569|that the be|gnifcance l|
|s||||||||s|Si|
|Measure|ML_b_||0.218_∗∗∗_|(0.047)||No|3,569|_b_ indicate|lications.|
||||_−_|||||ML|rep|
||CC||_−_0.069_∗_|(0.039)||No|1,098|and CC.|with 100|
||SR||0.148_∗_|(0.082)||No|3,569|vel for SR|bootstrap|
||ML_b_|0.219_∗∗∗_|(0.063)|||No|3,569|on area le|ained via|
||CC|0.056<br>|(0.048)|||No|1,098|enumerati|ons are obt|
||SR|_−_0.194_∗_|(0.099)<br>eds per ha|abor per ha||fect?<br>No|s<br>3,569|ard errors clustered at the|errors for the ML predicti|
||||Se|h l||E|on|and|rd|
|||ea|ty|f H||ed|ati|St|nda|
|||ar|ti|o||ix|v|_s:_|ta|
|||Plot|Quan|Days||EA F|Obser|_Note_|the s|

34
among other measures.

Second, in view of the complex logistics, supervision requirements, and ultimately higher costs associated with crop cutting, the question is whether crop cutting can be implemented on a sub-sample basis and if so, the minimum share of plot observations that should be subject to crop cutting in a way that allows us to estimate an imputation model that provides reliable measures of imputed CC yields. We assessed the predictive accuracy of our preferred ML specification with different sizes of the training data set - experimenting with 10, 20, 25, 50, 67, and 75 percent of the crop cut measures, while leaving the remaining sample in each scenario to be the test sample. It appears (see top panel of Figure 8) that conducting crop cutting on 50 percent of the plot observations may be sufficient for deriving reliable imputed CC yields. After the 50 percent mark, the gains in accuracy appear to be marginal.

Third, a related question is whether it is necessary to randomly designate a share of plot observations for crop cutting in all primary sampling units or whether reliable predictions can be obtained by working with a subsample of primary sampling units and conducting crop cutting on all plots in the designated areas. Since ML algorithms can perform well out of sample, logistics and supervision of crop cutting can potentially be simplified by limiting our footprint to fewer areas in a given country. To answer this question, we create an alternative training data set composed only of crop cut measures obtained in one randomly chosen ERIVaS stratum – as opposed to working with one-third of plots randomly chosen across all strata. The test sample in this case are composed of plot observations from the remaining ERIVaS strata. The bar plot in the bottom left panel of Figure 8 shows that the choice of the stratum has an important effect on the accuracy of the ML measure. The choice of the first stratum as the source of the training data results in an the _R_<sup>2</sup> value of 0.24 from the bivariate regression of observed CC yield against imputed CC yield in the test sample. The comparable _R_<sup>2</sup> estimate is 0.04 while working with the fourth stratum as the source of the training data. The reason behind these differences in performance might be the fact that the first stratum was the most heterogeneous unit in terms of CC yields: the kernel density of CC yields is more spread out within the first stratum vis-`a-vis the rest of the strata (see bottom right panel on Figure 8).

So, in view of national surveys that may consider implementing crop cutting on a sub-sample basis and deriving imputed CC yields for the rest of the sample, the findings imply that implementing crop cutting in a way that is confined to a particular region may only result in reliable predictions if the chosen region is sufficiently heterogenous in terms of crop yields.

35
**Figure 8:** Sampling strategies for optimal ML predictions. _Notes:_ The top panel shows the _R_<sup>2</sup> of theregression of CC on ML as a function of the training set size. We repeated the operation 50 times foreach sample size, so each point is the mean of the _R_<sup>2</sup> and the bars are the standard errors. The bottom panel illustrates how sampling unit choice af ects the out of sample _R_<sup>2</sup> . The bottom right panel shows the kernel density of the crop-cut yields in each sampling unit. For all simulations, we use the preferred ML model set up specification: we fit the model using log CC and include SR in the covariates.

<!-- Start of picture text --> 10% 20% 33% 50% 67% 75%<br>Training Set size (percent of total sample)<br>Bloc 1<br>Bloc 2<br>Bloc 3<br>Bloc 4<br>1 2 3 4 0 500 1500 2500<br>Bloc CC Yields (kg/ha)<br>0.25<br>2<br>0.20<br>ML Yield R<br>0.15<br>0.20 0.0012<br>2 0.15<br>0.0008<br>Density<br>0.10<br>ML Yield R<br>0.0004<br>0.05<br>0.00 0.0000<br><!-- End of picture text --> 36
# **6 Concluding remarks**

This paper attempts to increase the reliability of assessments of yield-input relationships when yields based on self-reported survey data on crop production are subject to non-classical measurement errors. We show analytically and empirically that it is possible to construct imputed measures of objective crop cut yields, using a machine learning model and a sub-sample of plot observations with both crop cut and self-reported yield measures in a household survey. The imputed crop cut yields are shown to be less biased vis-`a-vis self-reported yields, and the errors in the imputed measures are uncorrelated with input use per hectare. As such, the regressions of imputed crop cut yields replicate the true relationships between yields and inputs that are estimated with observed crop cut yields. These conclusions are robust to whether we use the data from a small-scale methodology study conducted in southwestern Mali or a nationally-representative survey.

From a data collection point of view, these findings carry the implications that more objective methods of collecting crop production data need to be integrated in the protocol of household surveys. We rely on two household survey data sets from Mali with the rare characteristic of having both self-reported and crop cut data. The first data set from a methodological experiment enables us to empirically show that it is possible to combine self-reported yields data to judiciously predict crop cut yields and, thus build a measure of yields that appears to be fit for a correct assessment of the relationship between yields and inputs with a random sample representing one-third of the entire sample. The second data set, a nationally representative survey, enabled us to validate the approach but also served as a stark reminder that even when a data collection method is theoretically objective, execution is key to ensure that meaningful inference can be made based on it. Although these results are promising, further work considering other crops should be pursued to improve the machine learning predictions, evaluate the limit of the inferences that can be carried out using this approach, and better understand its potential bias reducing properties.

Finally, our analysis expands on the body of work related to non-classical measurement errors and its impact on agricultural policy analysis in a smallholder context. We show that measurement errors in self-reported yields are correlated with major factors of production in smallholder farming systems. On land; the factor that has been the focus of the recent literature regarding the study of the inverse scale-productivity relationship (Desiere and Jolliffe, 2018; Abay et al.,2019; Gourlay et al.,2019), our results corroborate the existing evidence that IR disappears and is even reversed when the relationship is assessed with crop cut yield as opposed to self-reported yields. We also examine the relationships between yields and seeds per hectare and yields and household labor per hectare. These relationships have serious implications for agricultural policy especially as it pertains to the question of agricultural intensification. A positive relationship between yields and quantity of seeds suggests that farmers could improve their yields by applying more seeds and thus would provide support for making seeds more accessible to farmers. A positive relationship between yield and household labor indicates that plot-level land productivity could be increased with more labor, suggesting that labor is not correctly allocated across households. But similar to what is observed in the case of plot area, we document that measurement errors in self-reported yields are correlated with the quantity of seeds and household 37
labor used on the plot and that these errors lead to spurious relationships between yield and inputs use per hectare. We find that the positive relationship between yield and quantity of seeds is weakened or disappears, and the positive relationship between yield and household labor ceases to exist. Although our findings cannot be interpreted as causal and our measures of quantity of seed and labor need to be improved, these pieces of descriptive evidence suggest that intensification may not necessarily be the approach to follow to improve yields for smallholder farmers and that further research is needed in the domain.

38