After qualitative piloting, an extended version of the skill questionnaire was piloted in November 2013 on 120 farmers from an area in Siaya, close to the study area, and on farmers that had been selected in a similar way as those of the actual study population. A small subset of these farmers was also retested in December 2013 with the same survey instrument, in order to obtain retest statistics of the pilot. Based on this quantitative pilot, we eliminated questions with little variation.<sup>11</sup> We also removed questions that showed negative correlations with other variables meant to capture the same latent trait, and fine-tuned phrasing and translation of questions.<sup>12</sup> The final survey instrument took about 2.5 hours to complete.

The vast majority of farmers in the sample (97%) were native Luo speakers (the local language) – the others were more comfortable in Swahili or English (Kenya’s two official languages). The English-language survey therefore was translated in both Luo and Swahili. All versions were homogenized after independent back translation.<sup>13</sup>

# **2.3. Alternative measures of skills**

Prior to the set of questions in the three main modules described above, respondents were asked their self-assessment for the same set of skills using a set of 14 questions, formulated to proxy the different subdomains captured by the questions in the main modules. And after answering all questions from the three main sections, each farmer was asked to assess the skill level of one of the other farmers of his village in the sample using similar proxy questions. This provides an independent (though clearly subjective and possibly mis-measured) assessment. A second proxy measure comes from asking the same questions to another household member (typically the spouse) also involved in farming. And a third independent measure was obtained prior to the survey from the village health worker, who was asked to classify each farmer according to his

> 11 For instance, experience with pesticides or irrigation is extremely limited in the population of study, so that any related questions did not provide variation.

> 12 For the noncognitive module, a relatively large set of questions was identified with either very little variation (because everybody agreed with a certain positive statement), or a bi-modal distribution, typically in the case of reverse-coded questions. In extreme cases this led to negative correlations between variables that should capture the same latent trait.

> 13 Back-translation initially revealed a substantial number of questions with translation problems, in particular in the noncognitive part. As questions in this section are more abstract and referring often to concepts that are not part of daily vocabulary, finding the appropriate translation was often a challenge. For all sections, translations and backtranslations were compared, and we worked together with native Luo and Swahili speakers to finalize translations, to assure that the original meanings of the questions were maintained (and hence to know which questions we are in fact testing). We suspect that similar translation issues affect other surveys trying to obtain answers related to more abstract concepts, including some of the questions that are commonly used in the literature, which would need to be taken into account for the use of such measures.

10