# 4. Issues of measurement

## 4.1. Questionnaire design, seasonality, and survey implementation

Agricultural data collection often requires complex instruments reflecting the nature of agricultural production, with seasonality and a high prevalence of shocks as well as different input decisions taken with varying frequencies and at different times. Integrated agricultural surveys, like the LSMS-ISA, 50x2030 Initiative, and FAO’s AGRISurvey, collect data ranging from livestock production and asset ownership to agricultural inputs and labor use, and crop harvest quantity and value, among other issues. In some cases, input and output data are collected at the plot-level and multiple crops and seasons are covered in a single questionnaire (Dillon et al., 2021). The basic principles of questionnaire design apply as much to phone surveys as to other modes of agricultural data collection. However, phone surveys are considered more constrained than in-person surveys in terms of questionnaire length and complexity.

As we argue in this section, it is not realistic to administer such complex and extensive survey questionnaires over the phone in full. These operations will continue to require in-person interviewing. However, shorter special purpose survey questionnaires can be (and have been) administered in full by phone. Under certain conditions, phone surveys can complement in-person interviewing even for extensive agricultural surveys: some survey modules can benefit from the flexibility in timing that phone interviews offer, and some survey modules lend themselves more readily to data collection over the phone. Phone surveys can also provide an opportunity to collect additional information of interest.

### _Questionnaire length and complexity_

There is consensus that phone interviews should not be too long and recent research documents how response fatigue decreases data quality as the duration of phone surveys gets longer (Abay et al., 2021; Ballivian et al., 2015; Ceballos et al., 2020). Acceptable survey length varies depending on the topic, the country context, whether there is a previous relationship with the respondents, and on the respondents themselves. Glazerman et al. (2020) recommend interviews should not exceed 20 minutes while Mathur (2020a) find that the ideal survey length in that context was roughly 10–15 minutes. The LSMS-supported HFPS were designed to be around 20 minutes on average, though there was some variation across rounds and between countries, some of which averaged around 30 minutes (Gourlay et al., 2021). Survey designers will need to pilot questionnaires to determine what survey length is feasible without jeopardizing data quality and nonresponse.

In addition to length, questionnaire complexity is a limiting factor in phone surveys, which is thought to increase response fatigue and, in turn, affect data quality and attrition. Excessively complex questions should therefore be avoided in surveys of any mode. However, key concepts in agricultural statistics are inherently complex. Some complexity is necessary for survey questionnaires to reflect this adequately as oversimplification comes at a loss of information that can reduce the value of the data. In in-person surveys, interviewers rely on visual aids to make difficult questions more palatable. This is not possible with phone surveys.<sup>7</sup> Given these limitations in terms of length and complexity, certain topics of agricultural data cannot be collected over the phone. There are, however, some experiences with simplifying questionnaires to make

> 7 An alternative to be explored for the future may be the use of smartphone applications that provide visual aids and other survey support. An early example, Daum et al. (2019), used a pictorial smart-phone app to collect time use and nutrition data in rural Zambia.

14