We are also cognizant that there are trade-offs between reducing survey errors or increasing survey quality, and survey costs (Dillon et al., 2020). The most accurate measurement methods can be expensive and can come into direct competition with other survey objectives for example with covering a larger sample or collecting more information. Cost considerations are an important aspect in assessing how phone surveys can best complement or replace parts of in-person agricultural data collection. We discuss cost considerations in Section 5, highlighting the interactions and trade-offs between survey errors and survey costs.

## 2.2. Recent experience of agricultural and rural phone surveys in low-income settings

We draw from the experiences of several phone survey efforts implemented in rural areas, some of which were focused explicitly on agriculture, including: the World Bank-supported High Frequency Phone Surveys (HFPS), with an emphasis on the seven HFPS survey programs supported by the LSMS team in Burkina Faso, Ethiopia, Malawi, Mali, Nigeria, Tanzania, and Uganda (Living Standards Measurement Study, 2022); the Innovation for Poverty Action (IPA) RECOVR survey<sup>4</sup> ; the World Food Programme’s mVAM Project (World Food Programme, 2020); the Young Lives at Work program (Young Lives, 2022); the International Food Policy Research Institute (IFPRI) phone surveys (Alvi et al., 2021; Hirvonen et al., 2021a; Hirvonen et al., 2021b; Minten et al., 2020); World Bank’s Listening to Africa and Listening to Latin America and the Caribbean initiatives (Ballivian et al., 2015; Dabalen et al., 2016); a World Bank Gender Innovation Lab survey of women in agricultural households in Western Uganda (Sharma et al., 2021); and the recent Georgia Survey of Agricultural Holdings (50x2030 Initiative, 2020; FAO, 2018). Appendix I includes a brief description of these surveys.

We also draw on experiences from phone surveys used in regions affected by conflicts or natural disasters (Hoogeveen and Pape, 2020) and during the 2014-2016 Ebola epidemic in West Africa (Etang and Himelein, 2020; Himelein et al., 2015; Maffioli, 2020; World Bank, 2014; Zafar et al., 2016). Several survey experiments involving the use of phone surveys aimed at measuring agricultural labor and crop production (Arthi et al., 2018; Gaddis et al., 2021; Kilic et al., 2021) also provide valuable insights for the administration of agricultural surveys via Computer Assisted Telephone Interviewing (CATI). Finally, we draw lessons from several review articles on phone surveys conducted in low- and middle-income countries, including Dabalen et al., 2016; Dillon, 2012; Etang and Himelein, 2020; Glazerman et al., 2020; Gourlay et al., 2021; Henderson and Rosenbaum, 2020.

# 3. Issues of coverage and nonresponse

## 3.1. Sampling and representativeness

In phone surveys focusing on agriculture, the survey design problem when it comes to sampling and representativeness is reaching a representative sample of the target population of agricultural households, holdings, or individual farmers over the phone. Many of the challenges are common to any survey, but a few are specific to the phone survey mode (phone ownership or access, the existence or ability to generate a list of phone numbers, the difficulties of securing respondents’ consent at a distance) and to the fact that agricultural subjects (and in particular a non-random portion of them) are often less reachable by phone than the general population. In this section we illustrate how some of these issues play out in practice and the way and extent to which recent survey efforts have been able to overcome these issues to produce representative national estimates.

> 4 - - For more on IPA’s RECOVR Survey, visit: https://www.poverty <u>action.org/recovr/recovr survey.</u>

5