metadata: authors: 0: author: Kilic,Talip
1: author: Van den Broeck,Goedele
2: author: Koolwal,Gayatri B.
3: author: Moylan,Heather G.
count: Malawi
docna: 0: docna: Are You Being Asked ? Impacts of Respondent
            Selection on Measuring Employment
docty: Policy Research Working Paper
subtopic: Wages, Compensation & Benefits,Food Security,Livestock and Animal Husbandry,Gender and Development,Climate Change and Agriculture,Crops and Crop Management Systems
teratopic: Social Protections and Labor,Gender,Agriculture
abstracts: cdata!: Accurate estimates of men's and
            women's employment are at the heart of understanding
            sources of productivity and economic growth and designing
            well-targeted, gender-sensitive labor policies.  How
            respondent selection in household and labor force surveys
            affects these estimates is a key question, for which
            experimental evidence outside of high-income settings is
            limited. Leveraging two concurrent, national surveys in
            Malawi that differed in their approach to respondent
            selection, the analysis shows that compared to the best
            practice of privately interviewing adults about their
            employment outcomes, the common
            "business-as-usual" approach that permits the use
            of proxy respondents and non-private/group interviews leads
            to significant underreporting of employment across a range
            of wage- and self-employment activities, with stronger
            effects for women and for a longer (12-month) recall period.
            Under the business-as-usual approach, the main factors
            linked to under-reporting include household wealth, proxy
            reporting, and potential difficulties associated with
            interpreting/answering on questions regarding household
            non-farm enterprises.
geo_reg_key: 517195,517191
repnme: repnme: Are You Being Asked ? Impacts of Respondent
            Selection on Measuring Employment
display_title: Are You Being Asked Impacts of
            Respondent Selection on Measuring Employment
listing_relative_url: /research/2020/02/31792624/being-asked-impacts-respondent-selection-measuring-employment
topicv3: Agriculture and Food Security m1326205 435,Crop Production m1326627 174,Gender m1327621 615,Wage Employment m1504005 2703,Livestock m1504490 4024
publishtoextweb_dt: 2020-02-18T00:00:00Z
historic_topic: Social Protections and Labor,Gender,Agriculture
origu: Development Data Group (DECDG)
no_of_pages: 35
author_keys: 0: author_key: 886702
1: author_key: 788145
2: author_key: 1156566
model_extractions: Policy Research Working Paper 9152

# Are You Being Asked?

Impacts of Respondent Selection on Measuring Employment _Talip Kilic Goedele Van den Broeck Gayatri Koolwal Heather Moylan_ Development Economics Development Data Group February 2020
Policy Research Working Paper 9152

# **Abstract**

Accurate estimates of men’s and women’s employment are at the heart of understanding sources of productivity and economic growth and designing well-targeted, gender-sensitive labor policies. How respondent selection in household and labor force surveys affects these estimates is a key question, for which experimental evidence outside of high-income settings is limited. Leveraging two concurrent, national surveys in Malawi that differed in their approach to respondent selection, the analysis shows that compared to the best practice of privately interviewing adults about their employment outcomes, the common “business-as-usual” approach that permits the use of proxy respondents and non-private/group interviews leads to significant underreporting of employment across a range of wage- and self-employment activities, with stronger effects for women and for a longer (12-month) recall period. Under the business-asusual approach, the main factors linked to under-reporting include household wealth, proxy reporting, and potential difficulties associated with interpreting/answering on questions regarding household non-farm enterprises.

This paper is a product of the Development Data Group, Development Economics. It is part of a larger effort by the World Bank to provide open access to its research and make a contribution to development policy discussions around the world. Policy Research Working Papers are also posted on the Web at http://www.worldbank.org/prwp. The authors may be contacted at tkilic@worldbank.org.

_The Policy Research Working Paper Series disseminates the findings of work in progress to encourage the exchange of ideas about development issues. An objective of the series is to get the findings out quickly, even if the presentations are less than fully polished. The papers carry the names of the authors and should be cited accordingly. The findings, interpretations, and conclusions expressed in this paper are entirely those of the authors. They do not necessarily represent the views of the International Bank for Reconstruction and Development/World Bank and its affiliated organizations, or those of the Executive Directors of the World Bank or the governments they represent._ Produced by the Research Support Team
**Are You Being Asked? Impacts of Respondent Selection on Measuring Employment**

Talip Kilic<sup>‡</sup> , Goedele Van den Broeck<sup>ǂ</sup> , Gayatri Koolwal<sup>§</sup> , Heather Moylan*<sup>1</sup>

**Keywords:** Gender, Labor, Respondent Selection, Household Surveys, Malawi, Sub-Saharan Africa.

> 1 ‡Senior Economist, Living Standards Measurement Study (LSMS), Data Production and Methods Unit, Development Data Group (DECDG), The World Bank, Rome, Italy; tkilic@worldbank.org.<sup>ǂ</sup> Assistant Professor, Earth and Life Institute, UC Louvain, Louvain-la-Neuve, Belgium, <u>goedele.vandenbroeck@uclouvain.be</u> .<sup>§</sup> Consultant, LSMS, Data Production and Methods Unit, DECDG, The World Bank; <u>koolwalg@devscience.org.</u><sup>*</sup> Survey Specialist, LSMS, Data Production and Methods Unit, DECDG, The World Bank, Rome, Italy; hmoylan@worldbank.org. The authors thank Kathleen Beegle for her comments on the earlier draft of this paper.
# **1. Introduction**

Accurate estimates of men’s and women’s employment are at the heart of understanding sources of productivity and economic growth and designing well-targeted, gender-sensitive labor policies. Country employment surveys, spanning labor force surveys and labor modules in multi-topic household surveys, are usually structured to ask working-age individuals to report the incidence of and time allocation to a range of employment-related activities over different reference periods, for example over the last week or 12 months. The common “business-as-usual” approach in these surveys, however, can involve interviewing respondents in group settings; proxy respondents reporting on behalf of individuals that are not available for interviews; and other issues affected by survey time constraints, including lack of follow-up questions linked to difficult-to-measure or seasonal activities, as well as intra-household roles in enterprises. All of these variations can potentially create errors in reporting employment, particularly when questions involve longer recall periods, or for types of employment that are more difficult to measure, including seasonal/informal work. A key question, as a result, is whether individual interviews with respondents — focusing on self-reporting and one-on-one (as opposed to group) settings — have a significant effect on reporting of employment over the standard practice.

This study aims to answer this question by leveraging two national surveys that were implemented concurrently by the Malawi National Statistical Office with different approaches to respondent selection in collecting individual-disaggregated survey data, including on employment — and that allow us to compare the data obtained under the business-as-usual approach (the Fourth Integrated Household Survey (IHS4) 2016/17) versus individual interviews that are conducted in private (the Integrated Household Panel Survey (IHPS) 2016). As compared to individual interviews conducted in the IHPS, our findings show that the business-as-usual approach leads to significant underreporting of employment in livestock-related activities and household non-farm enterprises (in management and worker capacity) – with stronger effects for women and under a recall period of 12 months, as opposed to 7 days. The analysis also reveals underreporting in wage employment among men under the business-as-usual approach. Looking at how the choice of survey approach affects time allocation to employment, we find that most differences arise within agriculture, with the business-as-usual approach underestimating weekly hours spent by men and women in livestock activities as well as weekly hours devoted by men to crop agriculture. We find that the higher incidence of proxy reporting in the IHS4 is likely a significant contributor of underreporting, as well as potential difficulties, given less time spent with enumerators, that the IHS4 respondents may have faced in interpreting and reporting on concepts related to businesses or enterprises (as evidenced by greater discrepancies in reporting in the IHS4 across the labor and non-farm enterprise modules). Finally, household wealth is significantly associated with the extent of reporting differences across the individual interview and business-as-usual approaches.

2
The paper is organized as follows. Section 2 presents the literature review, followed by the description of the country context and data sources in Section 3. Section 4 lays out the empirical strategy and presents the results, and Section 5 concludes.

# **2. Literature Review**

Across high-income countries, there is a longstanding literature on respondent selection and labor reporting in nationally-representative surveys. In the U.S., for example, a large number of survey experiments have been conducted to better understand sources of survey measurement error in earnings by matching individual survey data with a concurrent, “validation” dataset (often administrative data from the Internal Revenue Service (IRS) or Social Security Administration, or employer’s records). Bound et al. (2001) provide a review of these validation studies, from 1950 up until 1999.

Further, several recent studies have matched individual-level survey rounds from the U.S. Census Bureau’s Survey of Income and Program Participation (SIPP) with administrative tax records from the IRS, to understand different sources of measurement error in earnings reporting. This includes proxy response, which can reduce survey costs and potentially provide less biased information on sensitive topics,<sup>2</sup> but also carries the risk of inaccurate reporting due to informational gaps, or differences in preferences and motivation between the proxy and intended respondent (see Cobb, 2018, and Bound et al., 2001). Cognitive issues related to respondents’ interpretation of questions, and ability to recall past outcomes accurately, additionally need to be considered (see Moore et al., 2000, and Bickart et al., 1990).

Tamborini and Kim (2013), for example, match the SIPP and IRS tax records at the individuallevel to find that proxy reporting leads to significant underreporting of earnings for single female workers (also see Hill, 1987). Cristia and Schwabish (2009) demonstrate that other covariates (in particular, earning higher incomes) lead to underreporting in the SIPP, and that individual characteristics that are associated with higher earnings (including age and marital status) are systematically associated with underreporting, leading to bias in estimates of these commonlyused predictors of earnings. Abowd and Stinson (2013) do not treat either approach as a “gold standard,” showing that both sources of data have measurement errors and argue for constructing a hybrid measure of earnings from survey and administrative data. Quality issues with earnings data across survey and administrative sources have also been raised. In the U.K., Britton et al. (2019) find systematically lower reporting of earnings in a dataset linking administrative tax records with student loan data, as compared to the concurrent Labor Force Survey — likely due to a combination of low survey response in the LFS from low/irregularly-paid earners, as well as a

> 2 Some studies on health outcomes in high-income countries, for example, have shown that proxy reporting can provide more accurate and detailed information on other household members’ health, including fewer issues with social desirability bias (see Mathiowetz and Groves, 1985, for a discussion using data from the U.S. National Health Interview Survey).

3
higher share of self-employed workers with no income in the administrative data (perhaps due to underreporting to avoid tax obligations).

Methodological survey research on measuring labor outcomes outside of high-income settings have been less common. The quality and regularity of administrative data is one reason, also due in large part to the prevalence of informal and irregularly-paid work in these countries. Recent validation studies have involved smaller, more focused experiments where concurrent household surveys contrast alternative methodological approaches, or modules testing for different approaches are randomized within the same survey. Bardasi et al. (2011), for example, randomized the implementation of a “short” and “detailed” module in the Survey of Household Welfare and Labour in Tanzania, a survey of 1,344 households across seven districts in the country, to examine how employment statistics are affected by (a) the level of detail of questions and (b) self-reporting as opposed to using proxies. Their findings have strong gender implications; proxy response significantly lowers male employment in agriculture, but no effect for women (the proxy effect is mitigated when the spouse is selected as the proxy and when the proxy has some education). A few additional screener questions to identify work mitigates over-reporting of employment among women engaged completely in domestic work, as well as higher average weekly hours conditional on working, especially for men. Dillon et al. (2012) use the same data and experiment to study the effects of proxy reporting and question detail on child labor, and in Serneels et al. (2017) to examine the returns to education. In both of these studies, proxy reporting did not have an effect, but screening questions (i.e. a detailed versus short module) did matter. Finally, the findings around respondent selection can vary by context and experimental design — in a separate experiment from Ethiopia of 1,200 Fairtrade coffee households (Galdo et al., 2019), survey respondents were randomly selected to test, across three different seasons, the effects of proxy reporting on reporting of child labor (children themselves versus the household head or spouse). Proxy reporting was found to significantly underreport work of girls in agricultural settings, relative to children’s own reports.

Our paper builds on this emerging literature by testing whether implementing individual interviews makes a significant difference in the quality of labor reporting, using two concurrent, nationallyrepresentative surveys from Malawi. We also examine how individual and household characteristics vary with survey approach to affect labor reporting.<sup>3</sup> Large-scale studies on labor survey methods from developing countries are also critical, given changes in international definitions of work and employment under the 19<sup>th</sup> International Conference of Labour Statisticians (19<sup>th</sup> ICLS) in 2013 that, for the first time, expand the definition of “work” to include all paid and unpaid activity, but at the same time narrow the definition of employment strictly to

> 3 Other recent developing-country experiments have found that specific characteristics of respondents can matter in accuracy of estimates; recent studies surveying farm households on recall periods and labor reporting, for example, find that recall bias is lower among better-educated respondents (see Gaddis et al, 2019 for a study from Ghana, and Arthi et al., 2018, for findings from Tanzania).

4
paid work (ILO, 2013).<sup>4</sup> This restricted definition of employment means that seasonal and/or irregularly-paid jobs — which by recent ILO estimates across 110 countries constitute about 70 percent of employment in developing and emerging economies, with much of it in agriculture (ILO, 2018a)) — need to be elicited more carefully, so as not to be missed through standard survey approaches. A better understanding of issues around respondent selection and survey measurement is therefore critical, to better design policies that foster economic opportunities, along with monitoring SDG targets around small-scale agricultural and informal employment.<sup>5</sup>

# **3. Country Context and Data**

Malawi is a predominantly rural country situated in southeast Africa, with a little over half of its households below the national poverty line, and 20 percent living in extreme poverty (World Bank, 2019). Agriculture makes up 26 percent of the GDP and is a source of employment for 83 percent of Malawian households (Davis et al., 2017). A high share of households, particularly in rural areas, are engaged in seasonal economic activities, including _ganyu_ — the local term for shortterm, temporary, off-farm labor opportunities (Beegle et al., 2017).

Our analysis is informed by two surveys that were implemented concurrently by the Malawi National Statistical Office: the Fourth Integrated Household Survey (IHS4) 2016/17 and the Integrated Household Panel Survey (IHPS) 2016.<sup>6</sup> The IHS4 was a multi-topic, nationallyrepresentative, cross-sectional household survey that was conducted over the period of April 2016April 2017. The IHS4 followed the business-as-usual approach to individual-level labor data collection, which allows for (1) proxy respondents to be interviewed on behalf of unavailable respondents, and (2) any respondent, whether proxy or self, to be interviewed in group, non-private settings. The Integrated Household Panel Survey (IHPS), on the other hand, was the third wave of a multi-topic, national, longitudinal household survey that was implemented from April 2016 to January 2017. The IHPS attempted to interview each adult household member in private, as part

> 4 The ICLS, which meets every five years, affects how country labor force surveys — guided by the ILO — are designed and undertaken.

> 5 Relevant SDG targets include, for example, Target 2.3 to double the agricultural productivity and incomes of smallscale food producers by 2030, through secure and equal access to land, other productive resources and inputs, knowledge, financial services, markets and opportunities for value addition and non-farm employment; as well as Target 8.3 to promote development-oriented policies that support productive activities, decent job creation, entrepreneurship, creativity and innovation, and encourage formalization and growth of micro-, small- and mediumsized enterprises including through access to financial services.

> 6 The data, questionnaires and basic information document for the IHS4 2016/17 can be accessed here: <u>https://microdata.worldbank.org/index.php/catalog/2936. The data, questionnaires and basic information document</u> for the IHPS 2016 can be accessed here: https://microdata.worldbank.org/index.php/catalog/2939. Both the IHS4 and IHPS were implemented with the technical and financial assistance from the World Bank Living Standards Measurement Study – Integrated Surveys on Agriculture (LSMS-ISA). The implementation of the individual interviews as part of the IHPS was made possible by the technical and financial assistance from the World Bank LSMS Plus (LSMS+) initiative, which aims to improve in IDA countries the availability and quality of individualdisaggregated survey data on asset ownership, work and employment and entrepreneurship. For more information on the LSMS+, please visit: http://surveys.worldbank.org/lsms/programs/lsms-plus.

5
of a broader effort to operationalize the international best practices in collecting individualdisaggregated survey data on asset ownership and work and employment. The individual interviews were capped at four per household and it was ensured that the head of household and his/her spouse (if one exists) were among the interviewed individuals.<sup>7</sup> Within-household interviews were always administered in private and were attempted to be administered simultaneously and with a gender match-up between the enumerator and respondent. Appendix I includes the protocol for administering the IHPS individual questionnaire.<sup>8</sup> In comparing the two survey approaches, our prior is that self-reporting with one-on-one interviews would more accurately reflect labor outcomes among respondents. Our analysis focuses on individuals aged 18-64, for whom the information was elicited during the IHS4-IHPS overlapping fieldwork period of April 2016-January 2017.<sup>9</sup> To compare the IHS4 to the IHPS, we pool the data on 17,433 IHS4 respondents together with 4,377 IHPS respondents that self-reported and were interviewed in private. The combined sample includes 21,810 observations. The IHS4 sample is a mix of self-reporting individuals as well as individuals for whom the information was provided through a proxy. The latter set constitutes 39.4 percent of the IHS4 sample.<sup>10</sup> We do not discard any IHS4 observations on the basis of self- versus proxy reporting since the official statistics would be based on the totality of these survey data, irrespective of the deviations from the international best practices.

Conversely, there are 969 IHPS respondents that were eligible but were unavailable for the individual interview, constituting a non-response rate of 16.8 percent (of a total of 5,776). To use the IHPS dataset in a way that can gauge the accuracy of the IHS4 data, we focus only on the IHPS individuals that have been subject to individual interviews and correct for non-response. To do the latter, we calculate weights by first running a logistic regression of individual response status among the 5,776 adults eligible for individual interviews. The results from the logistic regression are presented in the Appendix Table A1.<sup>11</sup> Subsequently, we (1) take the inverse of the predicted

> 7 This was an upper limit that only applied to one percent of the sampled household population that had more than four adults. If a sampled household had more than four adult household members, following the preference given to the head of the household, and his/her spouse if applicable, the remaining interview targets (2 or 3 depending on the presence of a spouse) were selected at random from the remaining pool of adult household members.

> 8 For more information on the organization and implementation of the individual-disaggregated data collection as part of the IHPS, please consult the survey’s basic information document, which can be accessed here: <u>https://microdata.worldbank.org/index.php/catalog/2939/download/47216.</u>

> 9 In view of seasonality of work and employment activities in Malawi, the sample is limited to the overlapping fieldwork months to ensuring temporal comparability of the individual-level data solicited in both surveys. 10 Comparing self-reporters versus individuals reported by proxy in the IHS4, we find that self-reporters tend to be significantly older and less educated, more likely to be female, married, to be the household head or spouse, and to live in smaller, poorer households with a higher dependency ratio than respondents who did not report for themselves (results available upon request). These differences are also in line with other studies that compare self-respondents and respondents-by-proxy (Bardasi et al., 2011; Serneels et al., 2017).

> 11 Van den Broeck and Kilic (2019) use a similar approach to correct for attrition bias in panel data samples, in a study of labor market dynamics in sub-Saharan Africa. Our right-hand-side predictors of “response” include (i) fixed effects for districts, interview months and enumerators, (ii) individual covariates, including age; a dichotomous variable

6
response probability to construct the response weight variable for each IHPS adult household member that was interviewed in private; (2) winsorize the response weights at the top 3 percent to account for potential outliers; and (3) set it equal to 1 for all adults in the IHS4 sample. Henceforth, all statistics are weighted using the response weight.

Table 1 presents the descriptive statistics on selected individual and household attributes. Individuals in the IHS4 are slightly older, more likely to be the household head or spouse, female and less educated, and they live in smaller households. Although these are statistically significant differences, the magnitude of the differences is negligible.<sup>12</sup>

**<u>Table 1. Descriptive Statistics by Survey Approach</u>**

|IHPS<sup>(1)</sup>|IHS4<sup>(2)</sup>|T-test<br>(1)–(2)|
|---|---|---|
|Age (years)<br>33.01|33.46|**|
|Female<br>0.52|0.53|**|
|Never attended school<br>0.09|0.13|***|
|No primary degree<br>0.52|0.51||
|Primary degree<br>0.12|0.12||
|Lower secondary degree<br>0.12|0.12||
|Higher secondary degree<br>0.15|0.12|*|
|Married<br>0.68|0.66||
|Household head or spouse<br>0.73|0.76|**|
|Log household size<br>1.61|1.52|***|
|Dependency ratio<br>0.89|0.93||
|Wealth index<br>0.10|0.16||
|Observations<br>4,377<br>**Notes:**IHPS = individual interviews; IHS4 =|17,433<br>business-as-usual|survey|

**<u>Notes:</u>** IHPS = individual interviews; IHS4 = business-as-usual survey approach. Significant differences are indicated with * (p<0.1), ** (p<0.05) and *** (p<0.01). Estimates are weighted with the response weight.

Table 2 presents the descriptive statistics for labor market outcomes of interest, by gender. The estimates are coupled with the results from the tests of mean differences across the two surveys.

identifying females; a series of dichotomous variables on educational attainment; dichotomous variables identifying whether the individual is currently married, and separately, whether he/she is head/spouse of head; and individual’s number of months living away from the household over the past year; and (iii) household covariates, including household size, dependency ratio, and wealth index. The latter is a factor analysis-based index that is composed of (i) a series of dichotomous variables that capture the ownership of mortar, bed, table, chair, fan, air conditioner, radio, radio with flash drive/micro CD, TV, VCR, sewing machine, kerosene/paraffin stove, electric/gas stove, refrigerator, washing machine, bicycle, motorcycle, car, minibus, lorry, beer-brewing drum, sofa, coffee table, cupboard/drawers, lantern, desk, clock, iron, computer, satellite dish, solar panel and generator, and (ii) a series of dwelling covariates, including number of dwelling rooms per capita and categorical variables that identify construction material (permanent; semi-permanent; traditional); roof type (grass; iron sheets; clay tiles/concrete/plastic sheeting/other); floor type (sand; smoothed mud; smooth cement/wood/tile/other); water source (piped/well; borehole; other), and toilet facility (flush/VIP toilet; traditional latrine; other/none).

12 These differences are due to the survey setting, as the IHS4 aims to represent the national population of Malawi in 2016, while the IHPS is based on the national population in 2010.

7
First, we analyze a range of dichotomous variables regarding participation in specific activities during the past seven days and during the past 12 months – constructed from the answers given to the screening questions asked at the beginning of the labor module to account for participation in the following activities:

- (1) **crop production** _(i.e. household farming activities whether for sale or for household food),_

- (2) **raising livestock** _(i.e. household livestock activities whether for sale or for household food),_

- (3) **managing a family non-farm enterprise (NFE)** _(i.e. run or do any kind of non-agricultural or non-fishing household business, big or small, for yourself),_

- (4) **working in an NFE** _(i.e. help in any of the household's non-agricultural or non-fishing household businesses),_

- (5) **engaging in off-farm wage employment** _(i.e. any work for a wage, salary, commission, or any payment in kind, excluding ganyu),_ and

- (6) **_ganyu_ employment** _(i.e. engage in casual, part-time or ganyu labor)._

Subsequently, we examine hours allocated to each of these activities during the past seven days, as well as annual hours allocated to and earnings from wage, and separately, _ganyu_ employment.<sup>13</sup> For now, we do not distinguish within the IHS4 between self-reporting respondents and respondents-by-proxy, although we do discuss proxy issues in the IHS4 later on. This is because the survey setting of the panel sample not only differs in terms of self-reporting but also in interviewing the respondents separately in absence of other household members, as explained above.

Table 2 shows that in the IHS4, all activities have the same or lower participation rates as compared to the IHPS, with the exception of activities in agriculture (any crop work — as well as men’s crop work strictly for household consumption). The distinction between crop production intended mainly for sale — versus all for household consumption/subsistence agriculture — was added to better understand how the survey approach affects the measurement of crop employment under the

> 13 Additional results including zeroes for the non-employed (i.e. unconditional sample) are available upon request. There were more significant differences in the unconditional sample owing to differences in participation across the two surveys. For weekly hours, we assume that no one can work more than 12 hours per day during the past seven days such that weekly hours are capped at 84 hours. We replace missing values with the median value of weekly hours within the enumeration area (EA) and take the logarithm conditional on participation. Annual hours are calculated based on the average number of hours per week spent on wage employment multiplied with the average number of weeks per month and the total number of months during the last 12 months. For _ganyu_ employment, we do not have data on the number of hours per week, so we assume an average of six hours per day multiplied with the number of days per week spent on _ganyu_ . We replace unrealistically high values with a maximum of 4,368 hours as we assume that no one can work more than 12 hours per day all year round. Annual earnings represent net income and are derived from all cash and in-kind remuneration that a worker received during the past 12 months. We winsorize earnings at top 1 percent; replace missing values with the median value within the enumeration area; and take the logarithm conditional on employment.

8
revised 19<sup>th</sup> ICLS definitions of work and employment. This has been a particularly challenging area for economies with a large share of smallholder farmers, for whom paid and subsistence work often vary seasonally throughout the year, and where women often have economically productive but hidden roles (Benes and Walsh, 2018a; Koolwal, 2019). Over the last few years, country labor force surveys and the surveys supported under the World Bank Living Standards Measurement Study – Integrated Surveys on Agriculture (LSMS-ISA) initiative have been introducing individual-level questions on the main intended destination of agricultural output to better elicit employment in agriculture under the new definitions.

Management of, and working in, an NFE and wage employment are also underreported under the IHS4 business-as-usual approach, and the difference widens when the recall period is longer. Although participation rates might differ, the ranking of importance of activities is similar in both scenarios, with cropping as the most common activity, then _ganyu_ and livestock, management of a non-farm enterprise, off-farm wage employment and self-employment as least common activities. Further, Table 2 shows that weekly hours worked among those employed are not statistically different across survey approaches for most activities, with some variations by gender (wage employment for men and _ganyu_ for women). This suggests that if participation is screened correctly, weekly hours tend to be adequately estimated for most activities under the business-asusual approach. For annual hours and earnings, we do find greater differences across survey approaches (among women, for example, conditional annual hours worked in wage employment are significantly higher in the IHS4, and conditional hours and earnings in _ganyu_ for both men and women are also significantly higher in the IHS4 as well). The choice of survey approach may matter more, for example, with longer recall periods as well as seasonality of different activities among this population.

Looking at gender differences, the last two columns of Table 2 show that women are significantly more likely than men under both survey approaches to work in crop agriculture over the last 12 months — particularly in own-use production as opposed to market activity — and less likely to work in wage employment and _ganyu_ . Weekly hours, on the other hand, tend to be greater for men across both farm and off-farm activities.<sup>14</sup> While men are significantly more likely to report management of a non-farm enterprise than women under the business-as-usual approach, this difference disappears in the individual-interview approach. Figure 1 looks at gender differences across samples by age, finding that the individual-interview approach in the IHPS tends to raise reported wage employment and NFE management for nearly all men across the age distribution, but for women the largest increases in these activities are focused on younger age groups (around 30 years of age; for NFE management there is an additional increase among older women (55+) in the IHPS as well). For women, lower time constraints outside of their child-rearing years may be one reason for these trends. Generally, Table 2 shows that men’s participation across different

> 14 Within crop work, hours spent separately in market activity versus own-use production were not collected (and also would be very difficult to collect).

9
off-farm activities rises in the IHPS relative to the IHS4 sample by a greater factor as compared to women, resulting in wider gender gaps for the IHPS sample.

|**Table 2. Compari**|**ng Individua**|**l-Level L**<br>|**abor Out**|**comes by S**|**urvey App**<br>|**roach and**|**Gender**<br>||
|---|---|---|---|---|---|---|---|---|
|||Men|||Women||Gender d|ifferences|
||IHPS<br>(1)|IHS4<br>(2)|T-test<br>(1)–(2)|IHPS<br>(3)|IHS4<br>(4)|T-test<br>(3)–(4)|IHPS<br>(1)-(3)|IHS4<br>(2)-(4)|
|**Labor activities (Y=1, N=0)**|||||||||
|Any crop work (7d)|0.26|0.32|**|0.29|0.35|**|-0.03|-0.03***|
|At least 1 crop mainly for sale (7d)<sup>†</sup>|0.42|0.39||0.33|0.35||0.09***|0.04***|
|All for HH own use (7d)<sup>†</sup>|0.44|0.53|***|0.59|0.58||-0.15***|-0.05***|
|Any crop work (12m)|0.69|0.72||0.74|0.75||-0.05***|-0.03***|
|Livestock (7d)|0.19|0.17||0.20|0.16|***|-0.01|0.01**|
|Livestock (12m)|0.33|0.30||0.31|0.29||0.02|0.01|
|Management of NFE (7d)|0.14|0.11|***|0.13|0.08|***|0.01|0.03***|
|Management of NFE (12m)|0.20|0.15|***|0.20|0.13|***|0.005|0.02***|
|Working in NFE (7d)|0.04|0.03|**|0.04|0.03|***|-0.001|-0.001|
|Working in NFE (12m)|0.09|0.05|***|0.08|0.06|***|0.01*|0.01**|
|Wage employment (7d)<sup>‡</sup>|0.18|0.15|**|0.06|0.06||0.12***|0.09***|
|Wage employment (12m)<sup>‡</sup>|0.22|0.18|**|0.08|0.07||0.15***|0.11***|
|_Ganyu_employment (7d)|0.22|0.23||0.17|0.16||0.05***|0.07***|
|_Ganyu_employment (12m)|0.47|0.46||0.36|0.37||0.11***|0.09***|
|**Log weekly hours, among employed**|||||||||
|Any crop work|2.39|2.27||2.19|2.24||0.40***|0.03*|
|Livestock|1.53|1.40|*|1.23|1.24||0.30***|0.16***|
|Management of NFE|3.12|3.04||2.73|2.74||0.39***|0.30***|
|Working in NFE|2.41|2.43||2.46|2.35||-0.05|0.08|
|Wage employment|3.53|3.44|**|3.24|3.37||0.29***|0.07*|
|Ganyu employment|2.56|2.47||1.94|2.11|***|0.62***|0.56***|
|**Log annual hours, among employed**|||||||||
|Wage employment|6.82|6.87||6.44|6.75|**|0.38***|0.12**|
|Ganyu employment|5.53|5.72|***|5.15|5.33|***|0.38***|0.39***|
|**Log annual earnings, among employed**|||||||||
|Wage employment|12.63|12.80||12.40|12.53||0.23|0.27***|
|Ganyu employment|10.72|10.90|**|10.02|10.16|*|0.70|0.74***|
|Observations<br>**Notes:**|1,966|8,163||2,411|9,270||||

IHPS = individual interviews; IHS4 = business-as-usual survey approach. Significant differences are indicated with * (p<0.1), ** (p<0.05) and *** (p<0.01). Estimates are weighted with the response weight.

† indicates that means are calculated among those involved in crop agriculture (1,215 individuals in the IHPS and 5,724 in the IHS4).

‡ Wage employment excludes _ganyu._ 10
**Figure 1.  Locally Weighted Regressions: Participation (Last 12 Months) in Off-Farm Activities, By Survey Approach and Gender**

<!-- Start of picture text --> NFE – management NFE - helping<br>Wage Ganyu<br><!-- End of picture text --> Moreover, Figure 2 depicts the kernel density distribution of conditional annual hours (top panel) and earnings (lower panel) across wage and _ganyu_ employment, for both men and women. Annual hours in wage employment for men and women tend to follow a bimodal distribution, indicating that people are either part-time or full-time wage employed, whereas hours in _ganyu_ employment tends to be unimodal, and focused around the lower concentration of wage employed (possibly reflecting, to some degree, that part-time wage employed may also be supplementing their income with _ganyu_ ). For women, the business-as-usual approach, relative to individual interviews, tends to lead to a higher concentration of individuals employed full-time, but a lower share among those working part-time/fewer hours. The distributions for men, on the other hand, indicate a higher share of both part-time and full-time wage employed (albeit a smaller increase among part-time workers/those with fewer hours). And regarding hours in _ganyu_ , the business-as-usual approach raises the concentration of men and women around the middle of their respective distributions.

11
Finally, Figure 2 is coupled with the p-values of the non-parametric Kolmogorov-Smirnov tests of equality of the IHPS and IHS4 distributions. The cross-survey differences are statistically significant in the comparisons of the distributions of (i) ganyu hours and earnings for women and men, and (i) wage hours for women. The IHPS distributions in these cases take on lower values vis-à-vis their IHS4 counterparts.

**Figure 2: Kernel Density Distribution of Log Annual Hours in and Log Earnings from Wage and Ganyu Employment, by Survey Approach and Gender**

**<u>Notes:</u>** Comparing the IHPS and IHS4 distributions, the K-S p-values were 0.026 for the Wage (F) distributions, 0.428 for Wage (M), 0.000 for Ganyu (F) and 0,001 for Ganyu (M).

**<u>Notes:</u>** Comparing across IHPS and IHS4, the K-S p-values were 0.616 for the Wage (F) distributions, 0.444 for Wage (M), 0.000 for Ganyu (F) and 0,001 for Ganyu (M).

12
# **4. Empirical Analysis**

# **_1. Effect of business-as-usual approach on labor reporting_**

To understand the effect of survey approach on labor reporting, we estimate the following equation separately for the samples of men and women aged 18-64: 1 + γ (1) iih 1iih iih yy = ∝ + ββ TT + εε Above, _i_ and _h_ represent individual and household, respectively; _y_ represents individuals’ labor outcomes (as presented in Table 2); and _α_ and _ɛ_ represent constant and error terms, respectively. 1 is a binary variable identifying the adults in the IHS4 sample, with the individuals in the IHPS constituting the comparison category. _D_ is a vector of individual and household attributes TT presented in Table 1 earlier to capture any remaining unobserved heterogeneity that may also jointly determine both the dependent variable and the likelihood of being selected for one survey versus the other.

The aim is to understand how the business-as-usual survey setting compares to the best-practice approach of conducting individual interviews with self-reporting respondents. For the binary outcome variables, Equation (1) is estimated as a linear probability model with probability weights adjusting for non-response.<sup>15</sup> For weekly hours across activities, and annual hours and earnings in wage and _ganyu_ , OLS regressions are run on the conditional sample of respondents (those who reported working in that activity over the relevant reference period).<sup>16</sup> Standard errors are clustered at the enumeration area level.

Based on equation (1), Tables 3 and 4 summarize the effect of the business-as-usual survey setting on the estimation of different labor statistics (for the full sample, and for men and women separately). On the whole, after controlling for demographic and survey characteristics, we find that the business-as-usual scenario (relative to the individual interviews) underestimates men’s and women’s participation in livestock, managing or working in an NFE, as well as wage employment for men. After controlling for demographic and survey characteristics, we find that the businessas-usual scenario significantly underestimates participation in the last 7 days in livestock, NFE management and supporting work (for both men and women), with weakly significant effects on wage employment for men. The effects are quite large, given participation rates in Table 2, and tend to be stronger for women. Among women, for example, Table 4 shows underreporting under the business-as-usual approach ranging from 3.2 percentage points for livestock (which, given mean participation for women in the IHS4, is a relative decrease of 20 percent); 3.9 percentage

> 15 The reported regression coefficients are qualitatively similar to the marginal effects derived from logit and probit regressions, which are available upon request.

> 16 Since the survey experiment/setting is exogenous, and we are focused on the conditional sample, we do not use a selection model in the analysis.

13
points for NFE management (a relative decrease of about 50 percent); and 1 percentage point for supporting work in an NFE (a relative decrease of 33 percent). For men, these relative decreases in reporting are 15 percent for livestock activity, 26 percent for NFE management, and 40 percent for supporting NFE work. The magnitude of the underestimation becomes even larger when the recall period changes to 12 months. There is no under- or over-estimation for cropping and _ganyu_ labor.

Looking at how the choice of survey approach affects time in employment, we find that most differences arise within agriculture. The business-as-usual approach underestimates weekly hours in livestock activities on the whole and for both men and women. Table 4 also shows that the business-as-usual approach also underestimates weekly hours for men in crop agriculture. We find that annual hours in _ganyu_ employment are higher under the business-as-usual as opposed to individual-interview approach for both women and men and reported earnings in _ganyu_ are higher for men. For women, we also find that annual hours in wage employment are higher under the business–as-usual approach.

14
**Table 3.  Estimates of the Impact of Business-as-Usual Survey Approach on** **<u>Labor Outcomes of All Adults Aged 18-64</u>**

|||**Within c**<br>**agricult**|<br>**rop**<br>**ure:**|<br>**Livestock**|<br>**Non-far**<br>**enterprise (N**|**m**<br>**FE):**|**Wage**|**_Ganyu_**|
|---|---|---|---|---|---|---|---|---|
||**A**|**A l 1**|**All**||||||
||**ny crop**<br>**work**<br>**(all HH)**|**t east**<br>**crop mainly**<br>**for sale**|**crops**<br>**for HH**<br>**own use**||**Ownership/**<br>**management**|**Helping/**<br>**supporting**|||
||**(1)**|**(2a)**|**(2b)**|**(3)**|**(4a)**|**(4b)**|**(5)**|**(6)**|
|**Total sample**|||||||||
|Participation (7d)|-0.015|0.007|0.017|-0.032 ***|-0.035 ***|-0.011 ***|-0.011|0.007|
||[-0.95]|[0.29]|[0.75]|[-2.93]|[-4.56]|[-2.69]|[-1.56]|[0.81]|
|R-squared|0.25|0.15|0.18|0.10|0.07|0.02|0.21|0.11|
|Participation (12m)|-0.013|-|-|-0.044 ***|-0.057 ***|-0.032 ***|-0.017 **|-0.004|
||[-1.09]|||[-3.38]|[-6.85]|[-5.15]|[-2.22]|[-0.39]|
|R-squared|0.38|-|-|0.16|0.08|0.04|0.23|0.22|
|_Observations_|_21,810_|_6,935_|_6,935_|_21,810_|_21,810_|_21,810_|_21,810_|_21,810_|
|Weekly hours (cond.)|-0.023|-|-|-0.145 ***|0.018|-0.169 *|-0.001|0.044|
||[-0.58]|||[-3.21]|[0.40]|[-1.67]|[-0.02]|[1.11]|
|R-squared|0.27|-|-|0.29|0.28|0.31|0.27|0.21|
|_Observations_|_7,026_|-|-|_3,745_|_2,215_|_718_|_2,291_|_4,249_|
|Annual hours (cond.)|-|-|-|-|-|-|0.069<br>[1.21]|0.177 ***<br>[4.14]|
|R-squared|-|-|-|-|-|-|0.25|0.18|
|Annual earnings|||||||||
|<br>(cond.)|-|-|-|-|-|-|0.070|0.134 ***|
||||||||[0.91]|[2.79]|
|R-squared|-|-|-|-|-|-|0.44|0.22|
|_Observations_|<br>-|-|-|-|-|-|_2,680_|_8,971_|
|**Notes:**|||||||||

The results are from linear probability models for binary outcomes, and from linear regressions for hours and earnings. Coefficients represent the impact of the business-as-usual survey approach, originating from regressions that also include the aforementioned control variables, and district, month and enumerator fixed effects. The regressions are weighted by the response weight, and t-statistics accounting for clustering at the enumeration-area level are presented in brackets. Significant differences are indicated with * (p<0.1), ** (p<0.05) and *** (p<0.01).

15
**Table 4.  Estimates of the Impact of Business-as-Usual Survey Approach on Labor Outcomes of Men and Women Aged 18-64**

|||**Within c**<br>**agricultu**|**rop**<br>**re:**|**Livestock**|**Non-f**<br>**enterprise**|**arm**<br>**(NFE):**|**Wage**|**_Ganyu_**|
|---|---|---|---|---|---|---|---|---|
||**Any crop**|<br>**At least 1**|<br>**All crops**||||||
||<br>**work**<br>**(all HH)**<br>**(1)**|**crop mainly**<br>**for sale**<br>**(2a)**|<br>**for HH**<br>**own use**<br>**(2b)**|**(3)**|**Ownership/**<br>**management**<br>**(4a)**|**Helping/**<br>**supporting**<br>**(4b)**|**(5)**|**(6)**|
|**Women**|||||||||
|Participation (7d)|-0.010|0.018|-0.023|-0.032 ***|-0.039 ***|-0.010 ***|-0.004|-0.007|
||[-0.51]|[0.75]|[-0.85]|[-3.05]|[-4.15]|[-2.25]|[-0.63]|[-0.69]|
|R-squared|0.27|0.16|0.19|0.10|0.07|0.03|0.17|0.12|
|Participation (12m)|-0.019|-|-|-0.034 **|-0.058 ***|-0.021 ***|-0.004|0.006|
||[-1.52]|||[-2.42]|[-5.70]|[-3.17]|[-0.63]|[0.48]|
|R-squared|0.38|-|-|0.16|0.10|0.05|0.20|0.23|
|_Observations_|_11,681_|_3,881_|_3,881_|_11,681_|_11,681_|_11,681_|_11,681_|_11,681_|
|Weekly hours (cond.)|0.069|-|-|-0.121 **|0.071|-0.109|0.094|0.147 **|
||[1.40]|||[-2.28]|[1.26]|[-0.84]|[1.12]|[2.57]|
|R-squared|<br>0.29|-|-|<br>0.32|<br>0.31|<br>0.40|<br>0.44|<br>0.21|
|_Observations_|_3,934_|-|-|_1,962_|_1,035_|_389_|_675_|_1,905_|
|Annual hours (cond.)|-|-|-|-|-|-|0.265**|0.162 ***|
||||||||[2.10]|[2.87]|
|R-squared|-|-|-|-|-|-|0.33|0.19|
|Annual earnings (cond.)|-|-|-|-|-|-|0.148|0.104|
||||||||[0.91]|[1.55]|
|R-squared|-|-|-|-|-|-|0.53|0.19|
|<br>_Observations_|-|-|-|-|-|-|_795_|_4,287_|
|**Men**|||||||||
|Participation (7d)|-0.019|-0.016|0.074 **|-0.026 **|-0.029 ***|-0.012 **|-0.021 *|0.024 *|
||[-1.06]|[-0.48]|[2.39]|[-2.01]|[-3.07]|[-2.06]|[-1.95]|[1.85]|
|R-squared|<br>0.23|<br>0.17|<br>0.21|<br>0.11|<br>0.07|<br>0.03|<br>0.23|<br>0.11|
|Participation (12m)|-0.005|-|-|-0.055 ***|-0.057 ***|-0.044 ***|-0.034 ***|<br>-0.014|
||[-0.37]|||[-3.63]|[-4.78]|[-5.00]|[-2.88]|[-0.92]|
|R-squared|0.39|-|-|0.17|0.09|0.05|0.24|0.22|
|_Observations_|_10,129_|_3,054_|_3,054_|_10,129_|_10,129_|_10,129_|_10,129_|_10,129_|
|Weekly hours (cond.)|-0.128 **|-|-|-0.191 ***|-0.054|-0.011|-0.069 *|-0.051|
||[-2.58]|||[-2.89]|[-0.93]|[-0.07]|[-1.75]|[-0.95]|
|_R-squared_|0.28|-|-|0.31|0.31|0.50|0.29|0.20|
|_Observations_|_3,092_|-|-|_1,783_|_1,180_|_329_|_1,616_|_2,344_|
|Annual hours (cond.)|-|-|-|-|-|-|0.006|0.197***|
||||||||[0.11]|[3.74]|
|R-squared|-|-|-|-|-|<br>-|0.26|0.16|
|Annual earnings (cond.)|-|-|-|-|-|-|0.07|0.163***|
||||||||[0.89]|[2.72]|
|R-squared|-|-|-|-|-|<br>-|0.45|0.20|
|_Observations_|-|-|-|-|-|-|_1,885_|_4,684_|
|**Notes:**|||||||||

The results are from linear probability models for binary outcomes, and from linear regressions for hours and earnings. Coefficients represent the impact of the businessas-usual survey approach, originating from regressions that also include the aforementioned control variables, and district, month and enumerator fixed effects. The regressions are weighted by the response weight, and t-statistics accounting for clustering at the enumeration-area level are presented in brackets. Significant differences are indicated with * (p<0.1), ** (p<0.05) and *** (p<0.01).

16
<!-- Start of picture text --> 0<br>Any crop Live: k Ni NFE: help/ Wage Ganyu<br>work own/| ge support employment<br>0.02<br>0.04<br>0.06<br>0.08<br>i Effect of IHS4 proxy vs. IHPS<br>-0.1<br>G Effect of IHS4 non-proxy vs. IHPS<br>0.12<br><!-- End of picture text -->

<!-- Start of picture text --> 0.06<br>0.04<br>0.020 \<br>Any crop Livestock NF NFE; help/ Ww Ganyu<br>-0.02 work own/m e support employment<br>-0.04<br>-0.06<br>-0.08 Effect of IHS4 proxy vs. IHPS<br>“04 © Effect of IHS4 non-proxyvs. IHPS<br>-0.12<br><!-- End of picture text --> <!-- Start of picture text --> 0.06<br>0.04<br>0.02<br>0<br>Any crop L ‘ock Ni NEE: / Wage yu<br>0.02 work own/! P| employment<br>0.04<br>0.06<br>0.08 @ Effect of IHS4 proxy vs. IHPS<br>0.1<br>Effect of IHS4non-proxy vs. IHPS<br>0.12<br><!-- End of picture text -->

<!-- Start of picture text --> 0.06<br>0.04 A<br>0.020 Z<br>Any crop work Livestoe NF NFE: Wi Ganyu<br>-0.02 own/m#hage —— sup} emplo'<br>-0.04<br>-0.06<br>-0.08 Effect of IHS4 proxy vs. IHPS<br>70.1 21 Effect of IHS4 non-proxyvs. IHPS<br>-0.12<br><!-- End of picture text -->
more detailed questioning to better capture this work. This mechanism helps to explain the results in Figure 3 showing that the IHS4 non-proxy sample reports lower non-farm enterprise and wage employment. Recent cognitive studies conducted by the ILO across countries, for example, have found that many respondents in highly informal and agricultural contexts can have difficulty — without detailed contextual and follow-up questions — on identifying their work in an enterprise or casual wage work (Benes and Walsh, 2018b).

Table 5 aims to better understand whether this mechanism is at work for non-farm enterprise activity, where underreporting under the business-as-usual approach is more systematic across the wealth distribution and for both men and women. Specifically, Table 5 compares the enterpriserelated outcomes above that come from the labor module, with the same outcomes from a separate household enterprise module that was also part of each survey. The enterprise module in each survey creates a roster of all enterprises that household members are involved in, and for each enterprise, one person (almost always the owner/manager, discussed below) reported on roles across household members on owners, managers, as well as those in supporting roles.

We find that the IHPS individual-interview approach tends to result in lower discrepancies between the labor module and non-farm enterprise module for the overall sample (although there is a significant difference, albeit small in magnitude, for the share of women working in a supporting role). For the IHS4, on the other hand, a significantly greater number of men and women are captured by the enterprise module as owners/managers, as well as supporting workers. Within the sample of households with an enterprise, there are significant differences in both the IHPS and IHS4, where the nonfarm enterprise module captures a greater share of men and women owners/managers as compared to the labor module. The discrepancy, however, is much greater under the business-as-usual approach, and also relatively greater for women (an increase from 37 to 51 percent in women’s ownership/management moving from the labor module to the nonfarm enterprise module, or a 14-percentage point increase) as compared to men (a 12-percentage point increase).

In addition, there were cases of proxy respondents in the enterprise module (those that neither owned/managed or worked in the enterprise), although for the sample of individuals used in the analysis, the share of proxy response was lower in the IHPS (6.7 percent, or 64 individuals) compared to the IHS4 (10.4 percent).<sup>17</sup> The last two columns of Table 5 show that proxy response in the non-farm enterprise module has a greater effect on reporting of ownership/management in the IHS4, particularly for men — where own-reporting tended to be lower than the overall (proxy plus own-reporting) sample. There was, otherwise, no effect of proxy response in the IHPS non-

> 17 The enterprise module provides a closer look at the types of enterprises that had a proxy respondent — nearly all (99 percent) of these enterprises had only one owner/manager, with most (about 80 percent) having no other supporting workers. These tended to be enterprises owned by the husband, with the spouse reporting. Among those enterprises with the owner/manager not reporting and which had workers, however nearly all (20 out of 25) of IHPS respondents were working in that enterprise, compared to 69 percent in the IHS4 (66 out of 96 enterprises).

18
farm enterprise module, and proxy response in the non-farm enterprise module did not have an effect on supporting work roles in either survey approach. Along with issues around interpreting one’s role in a business, therefore, proxy reporting might be another issue to consider, although larger differences in reporting seem to emerge across the labor and non-farm enterprise modules than by proxy response (as discussed as part of the related regressions below).

**Table 5.  Share of Individuals Running, and Providing Supporting Work in an NFE, by Survey Approach:** **<u>Comparisons across Labor and NFE Modules</u>**

|||<br>|||<br>||**NFE m**<br>**sample (p**<br>**s**<br>|**odule: c**<br>**roxy + se**<br>**elf-repor**<br>|**omparing ov**<br>**lf-reports)**<br>**ting sample**<br>|**erall**<br>**with just**<br> <br>|
|---|---|---|---|---|---|---|---|---|---|---|
|||**Women**|||**Men**||**Wom**|**en**|**M**|**n**|
||as reported in<br>labor module<br>(overall<br>sample)<sup>(1)</sup><br>|as reported<br>in NFE module<br>(overall<br>sample)<sup>(2)</sup>|T-test<br>(1)–(2)|as reported<br>in labor<br>module<br>(overall|as reported<br>in NFE<br>module<br>(overall|T-test<br>(3)–(4)|NFE<br>module:<br>only self-<br>reports<sup>(5)</sup>|T-test<br>(2)–(5)|NFE<br>module:<br>only self-<br>reports<sup>(6)</sup>|T-test<br>(4)–(6)|
|||||sample)<sup>(3)</sup>|sample)<sup>(4)</sup>||||||
|**Share of individuals running a**|||||||||||
|**business, in last year†**|||||||||||
|(A) IHPS/individual interviews|||||||||||
|Overall sample|0.20|0.18||0.20|0.18||0.17||0.17||
|_Observations_|_2,411_|_2,411_||_1,966_|_1,966_||_2,395_||_1,919_||
|Among HHs with enterprise|0.41|0.47|***|0.41|0.45|*|0.46||0.42||
|_Observations_|_899_|_899_||_778_|_778_||_883_||_731_||
|(B) IHS4/business-as-usual|||||||||||
|Overall sample|0.13|0.16|***|0.15|0.17|***|0.15|*|0.15|***|
|_Observations_|_9,270_|_9,270_||_8,163_|_8,163_||_9,175_||_7,924_||
|Among HHs with enterprise|0.37|0.51|***|0.42|0.54|***|0.50||0.49|***|
|_Observations_|_2,891_|_2,891_||_2,608_|_2,608_||_2,796_||_2,369_||
|**Share working in supporting role**|||||||||||
|<br>**in business, over most recent**<br>|||||||||||
|**survey period:**<sup>**(4)**</sup>|||||||||||
|(A) IHPS/individual interviews|||||||||||
|Overall sample|0.04|0.03|***|0.04|0.04||0.03||0.03||
|_Observations_|_2,411_|_2,411_||_1,966_|_1,966_||_2,395_||_1,919_||
|Among HHs with enterprise|0.10|0.08||0.09|0.10||0.07||0.07||
|_Observations_|_899_|_899_||_778_|_778_||_883_||_731_||
|(B) IHS4/business-as-usual|||||||||||
|<br>Overall sample|0.03|0.02|***|0.03|0.026|*|0.02||0.02||
|<br>_Observations_|_9,270_|_9,270_||_8,163_|_8,163_||_9,175_||_7,924_||
|Among HHs with enterprise|0.08|0.08||0.09|0.07|*|0.07||0.05||
|_Observations_|_2,891_|_2,891_||_2,608_|_2,608_||_2,796_||_2,369_||
|**Notes:**|||||||||||

Significant differences are indicated with * (p<0.1), ** (p<0.05) and *** (p<0.01). Estimates are weighted with the response weight.

† indicates that in the labor reporting module, the question asked of respondents was “in the last 12 months, did you run a non-farm business of any size for yourself or the household, even if only for one hour?” In the non-farm enterprise module (covering activity over the last 12 months), one respondent for each enterprise was asked to identify up to two household members for each of the following: (a) “who manages this enterprise or is most familiar with it,” and (b) “who owns this enterprise?” ‡ indicates that the most recent survey period in the labor module was the last 7 days, and in the NFE module was the last operational month. These two groups were more comparable than using the last 12 months in the labor module (which led to much higher rates of participation).

19
Figure 4 further looks at specific cases where individuals were identified as owners/managers of a business in one module, but not the other. Again, the nonfarm enterprise module captures a greater share of individuals as owners/managers than the labor module; although the number of cases is not large, we do see much wider gaps across modules in the IHS4 compared to the IHPS, and the labor module in the IHPS also tends to capture a greater share of individuals owning/managing an enterprise. Table A4 in the Appendix also runs regressions of individuals’ work in enterprises using data strictly from the enterprise module and finds very few differences by survey approach. Where there is significant underreporting under the business-as-usual approach, the magnitudes of the coefficients are smaller compared to Table 4. Overall, therefore, underreporting of enterprise/business participation under the business-as-usual approach is linked significantly to the level of detail of questions in the labor module, which the individual-interview approach helps address.

**Figure 4.  Discrepancies across Labor and NFE Modules in Identifying Individuals Who Own/Run an NFE for IHPS versus IHS4**

# **_3. Interaction effects of business-as-usual approach on labor reporting: the role of household wealth quintiles_**

To further understand whether under- and over-reporting in the business-as-usual scenario are stronger for certain subpopulations, Appendix Tables A5 and A6 summarize the interaction effects (on participation in the last 7 days; results were very similar for participation in the last 12 months) of the business-as-usual survey approach with education, relation to household head, age, household wealth and location. The results are presented separately for the samples of men and women and show that along with age (being in the younger 15-24 age group), interaction effects with household wealth are associated with a greater range of labor participation outcomes for men 20
and women.<sup>18</sup> For households with greater wealth under the business-as-usual approach, reporting of employment in market work in agriculture, as well as livestock, is significantly higher, but lower for non-farm enterprise work (ownership/management). On the other hand, the interactions of the business-as-usual survey approach with relation to the household head, marital status, and dependency ratio have relatively fewer significant effects across different labor outcomes.

Figure 5 examines how the business-as-usual approach affects reporting on participation in specific activities during the last 7 days _within_ each wealth quintile (i.e., estimating equation (1) separately for each wealth quintile). The trends in 12-month participation are similar (though with larger effects) and are presented in Appendix Figure A1.<sup>19</sup> As seen in Tables 3 and 4, Figure 5 shows that business-as-usual approach leads to under-reporting of labor participation across most activities, relative to individual interviews, with the exception of men’s employment in _ganyu_ , and with ambiguous effects on wage employment. Figure 5 shows that whether under-reporting is greater among higher or lower wealth quintiles depends on sector of activity. Similar to the results in the Appendix Tables A5 and A6, the business-as-usual approach leads to greater under-reporting in non-farm enterprise work within higher wealth quintiles, who tend to be focused in these activities — with larger effects for ownership/management as opposed to supporting work. Underreporting in agriculture, on the other hand, is more likely among men and women in lower wealth quintiles, who are also more likely to be mainly concentrated in agriculture.

Generally, trends in over/under-reporting tend to be similar for women and men for self-employed activities. For wage and _ganyu_ employment, however, there are cases where trends for men and women move in opposite directions, similar to the pooled sample results presented in the Appendix Tables A5 and A6. With wage employment, for example, we find significant over-reporting of men’s employment under the business-as-usual approach in the highest quintile — but for women, relative to individual interviews, the business-as-usual approach leads to under-reporting, even though full results from the regressions in Tables 3 and 4 (available upon request) show that wage employment is positively associated with higher wealth quintiles for both men and women. For _ganyu_ , there is significant over-reporting of men’s employment among the lowest wealth quintiles, but under-reporting among women under the business-as-usual approach. Given the earlier results from Tables 3 and 4 showing that underreporting under the business-as-usual approach tend to be greater for women, some of the gender differences may be due to norms around how women’s paid work outside the household is viewed, particularly when women themselves are not reporting. Women are often engaged in several productive activities within and outside the home, for example, but their main roles may continue to be viewed as domestic instead of in paid labor (see Benes and Walsh, 2018a; Fletcher et al., 2018; Comblon and Robilliard, 2017; and Fox and Pimhidzai 2013).

> 18 We also found that wealth interactions with the business-as-usual approach also tend to have more widespread effects on participation (in the last seven days or 12 months) as opposed to weekly/annual hours and annual earnings (results available upon request).

> 19 The full estimates are available upon request.

21
**Figure 5.  Effects of Business-as-Usual Approach on Labor Outcomes in the Last Seven Days by Wealth Quintile**

<!-- Start of picture text --> Any crop work Livestock<br>Non-farm enterprise: ownership/management Non-farm enterprise: supporting work<br>Wage employment Ganyu employment<br><!-- End of picture text -->

**<u>Notes:</u>**

The quintiles are for the wealth index, as explained in footnote 11. q1 = lowest wealth quintile, q5 = highest wealth quintile. The points in the graph represent coefficients from linear probability models of participation on the business-as-usual survey approach, for each wealth quintile, controlling for district, month and enumerator fixed effects. Lines represent 95 percent confidence intervals. The regressions are weighted by the response weight.

22
# **5. Conclusions**

Accurate measurement of employment is at the heart of designing effective policies that improve economic opportunities for the poor and alleviate gender inequities in access to and returns from productive activities. Leveraging two national surveys that were implemented concurrently in Malawi with different approaches to respondent selection in collecting individual-disaggregated survey data, we find that compared to the best practice of conducting private individual interviews to elicit self-reported information on labor outcomes, the business-as-usual approach that allows for proxy respondents and group/non-private interviews leads to significant underreporting of employment in livestock-related activities and household non-farm enterprises (in management and worker capacity). The effects are stronger for women and under a recall period of 12 months, as opposed to 7 days. The business-as-usual approach also brings about under-reporting in the incidence of wage employment among men.

We further discuss how underreporting is associated with a higher incidence of proxy reporting under the business-as-usual approach, as well as potential difficulties — given less time spent with enumerators — that respondents may face in interpreting and reporting on work that may be seasonal, such as casual wage work, as well as work in enterprises (as evidenced by greater discrepancies in reporting across the labor and non-farm enterprise modules under the businessas-usual approach, with more pronounced differences for women). The extent of under- and overreporting is also systematically associated with household wealth (greater under-reporting of nonfarm enterprise employment for the highest wealth quintile, for example), highlighting the importance of respondent selection on specific socioeconomic subgroups as well as by gender.

23