metadata: authors: 0: author: Ambler,Kate
1: author: Herskowitz,Sylvan Rene
2: author: Maredia,Mywish Karimbhai
count: Malawi
docna: 0: docna: Rural Labor and Long Recall Loss
docty: Policy Research Working Paper
abstracts: cdata!: Surveys frequently rely on annual recall
            to capture individuals&#x2019; labor activities over the
            preceding year. This paper uses a panel of rural households
            in Malawi for a survey experiment to test the effect of a
            long, annual recall window on reported labor supply relative
            to a set of quarterly interviews. The paper documents large
            losses in reported labor participation using the long recall
            window with reductions of over 20 percent of reported
            activities and months worked and a 2.5 times greater
            incidence of reported unemployment relative to the shorter
            window. These losses are greater for activities further in
            the past and especially for individuals whose labor supply
            is reported by other family members, reaching up to 50
            percent for some outcomes. The profile of
            households&#x2019; primary respondents, predominantly
            male and older, and differential effects by age further
            suggest that long recall may cause meaningful biases in the
            resulting data for women and younger household members.
display_title: Rural Labor and Long Recall Loss
listing_relative_url: /research/2024/01/34243842/rural-labor-long-recall-loss
topicv3: Poverty Measurement and Analysis m1326941 707,Household Surveys m1504071 2761,Labor Diagnostics m1583605 4430
publishtoextweb_dt: 2024-01-23T00:00:00Z
dois: 10.1596/1813-9450-10680
origu: Off of Sr VP Dev Econ/Chief Econ (DECVP)
no_of_pages: 34
author_keys: 0: author_key: NA
1: author_key: 1111373
2: author_key: 434944
model_extractions: Policy Research Working Paper 10680

# Rural Labor and Long Recall Loss

_Kate Ambler_ _Sylvan René Herskowitz Mywish K. Maredia_ Development Economics Development Impact Group January 2024 A verified reproducibility package for this paper is available at http://reproducibility.worldbank.org, click **here** for direct access.
Policy Research Working Paper 10680

# **Abstract**

Surveys frequently rely on annual recall to capture individuals’ labor activities over the preceding year. This paper uses a panel of rural households in Malawi for a survey experiment to test the effect of a long, annual recall window on reported labor supply relative to a set of quarterly interviews. The paper documents large losses in reported labor participation using the long recall window with reductions of over 20 percent of reported activities and months worked and a 2.5 times greater incidence of reported unemployment relative to the shorter window. These losses are greater for activities further in the past and especially for individuals whose labor supply is reported by other family members, reaching up to 50 percent for some outcomes. The profile of households’ primary respondents, predominantly male and older, and differential effects by age further suggest that long recall may cause meaningful biases in the resulting data for women and younger household members.

This paper is a product of the Development Impact Group, Development Economics. It is part of a larger effort by the World Bank to provide open access to its research and make a contribution to development policy discussions around the world. Policy Research Working Papers are also posted on the Web at http://www.worldbank.org/prwp. The authors may be contacted at sherskowitz@worldbank.org. A verified reproducibility package for this paper is available at http:// reproducibility.worldbank.org, click **here** for direct access.

_The Policy Research Working Paper Series disseminates the findings of work in progress to encourage the exchange of ideas about development issues. An objective of the series is to get the findings out quickly, even if the presentations are less than fully polished. The papers carry the names of the authors and should be cited accordingly. The findings, interpretations, and conclusions expressed in this paper are entirely those of the authors. They do not necessarily represent the views of the International Bank for Reconstruction and Development/World Bank and its affiliated organizations, or those of the Executive Directors of the World Bank or the governments they represent._ Produced by the Research Support Team
# **_Rural Labor and Long Recall Loss_**

Kate Ambler Sylvan René Herskowitz Mywish K. Maredia IFPRI World Bank Michigan State University

**Keywords:** Labor, Survey Methodology, Measurement Bias, Recall Windows, Proxy Response **JEL Codes** : O1, J2, C8, Q1

This work was supported by the CGIAR Research Program on Policies, Institutions, and Markets and was approved by the IFPRI Institutional Review Board and the Malawi Committee on Social Science and Humanities, National Commission for Science and Technology. We thank the MwAPTA Institute for collecting the baseline data, GeoPoll for data collection, and Laura Leavens for research support. The findings, interpretations, and conclusions expressed in this paper are entirely those of the authors. They do not necessarily represent the views of the World Bank and its affiliated organizations, or those of the Executive Directors of the World Bank or the governments they represent.
# **1. Introduction and Background**

Understanding peoples’ productive activities over the course of the year is important both to document how people earn their livelihoods and also to ensure that poverty programs are well designed and well-targeted. Measuring employment is especially challenging in settings where productive activities are informal, with irregular intensity of participation, and seasonal, where much of the effort and earnings are concentrated in specific periods of the year (Feuerbacher et al. 2020). These characteristics are particularly relevant in rural labor markets in low-income countries that rely heavily on employment in the agriculture sector (Fink, Jack, and Masiye 2020; World Bank, 2017).

The prevailing survey methodology literature considers shorter recall periods to be more accurate, often relying on a standard seven-day recall window (Durazo et al. 2021). However, relying on reported labor activities over the most recent seven days is likely to miss individuals’ full range of labor contributions and work activities over a full year, especially in highly seasonal and informal labor markets. To address this gap, standard data collection practices typically also include questions about productive activities of each household member over the past year.<sup>1</sup> While existing evidence is mixed, use of such a long window may lead to losses in data and data quality.<sup>2</sup> In this paper, we use a survey experiment to test the effect of recall windows on reported annual work behaviors, contrasting reported labor participation using a long (annual) recall method to those resulting from a set of quarterly interviews that follow a similar structure and set of questions, but over shorter, 90-day, recall windows.

A commonly used format for annual recall labor questions in multitopic household surveys asks the household’s primary respondent to report on their labor activities over the preceding year, including type of activity, periods worked, and intensity of participation. These questions are asked about their main activity over the year as well as any secondary activity. While there is often a stated preference for having each household member respond for themselves, frequently, either when that respondent is unavailable or when survey budget resources are insufficient to allow for

> 1 The most common format for large multi-topic surveys such as the World Bank’s Living Standards Measurement Study is to ask about activities in the last seven days in detail, with follow-up questions regarding the last 12 months (see Durazo et al. 2021).

> 2 See de Weerdt, Gibson, and Beegle (2020) for an excellent review of the survey methods literature that includes an overview of both recall windows and proxy reporting.
self-reporting by all household members, the household’s primary respondent will then report on behalf of other members as well, “proxying” their responses.<sup>3</sup> Recently, the global penetration of cell phone access has introduced phone surveys as a method of data collection that could allow for more frequent interview intervals along with shorter recall periods at a lower cost. While weekly interviews have been shown to be less vulnerable to recall loss than longer windows (Arthi et al. 2018), weekly interviews across an entire year or season would be prohibitively expensive for most research budgets and highly fatiguing for respondents. In this paper we use a survey experiment to test an intermediary option, comparing quarterly (90day) with an annual recall question using a series of phone surveys.

Labor outcomes measured using the long recall responses result in much lower recorded levels of any work involvement (22%), number of unique activities reported (24%), and number of months worked (20%) compared to those based on the quarterly measures, although we do not find significant losses in total hours worked. These gaps increase with the amount of time that has passed. Proportional losses from long recall (relative to short recall averages) are, depending on the outcome variable, between two and ten times larger for proxied individuals than for households’ primary respondents who self-report. Splitting the sample by self and proxy reported individuals, we explore heterogeneity by gender and age of the respondent as well as of the proxied household member. We do not see clear patterns of heterogeneity for self-reports or by the primary respondent’s characteristics when reporting for others. We do, however, see that younger household members have greater relative losses from long recall when their responses are proxied than older proxied household members. The profile of primary respondents (generally household heads) further biases annual recall data against women and younger household members, as they will be exposed to larger losses from long recall as a consequence of having their labor contributions more frequently reported by proxy.

This paper contributes to a growing body of work on the effects of recall periods in the survey methodology literature. A considerable body of evidence suggests that longer recall periods tend to undercount household consumption, provision of agricultural inputs, and negative health events

> 3 The Living Standards Measurement Study (LSMS) surveys follow a similar structure and have a similarly stated preference for self-reporting whenever possible. In a review of six LSMS surveys, Desiere and Costa (2019) find that the use of proxy reporting is still widespread, ranging between 24% in Nigeria up to 85% in Mali.
relative to shorter recall windows.<sup>4</sup> The evidence on recall windows as it applies to labor activities, however, is more mixed. In a study based in Tanzania, Arthi et al. (2018) compare reported contributions of farm labor in an agricultural plot-based module using weekly recall with those from an end of season recall window. Longer recall windows exaggerate the number of hours worked per person by a factor of four, although the reported number of active plots and individual level participation are undercounted. In our rural sample in nearby Malawi, we find reductions in reported labor contributions using the longer recall periods relative to a shorter recall window, but no overall difference in hours worked, the intensive margin on which Arthi et al. (2018) find positive effects. In urban labor markets, Heath et al. (2021) examine very short recall windows and find modest effects of switching from weekly to daily recall with higher reported self- employment spells but no impact on wage employment. However, Garlick et al. (2020) find that weekly versus monthly surveys did not influence data quality or reported microenterprise activities.

This paper also links to a second area of the methods literature centered on the use of proxy responses for data collection. Of note, Bardasi et al. (2011) find that males are more affected by proxy losses in reported agricultural labor activities than women, though this is reduced when the primary respondent is his wife or well-educated. By contrast, Serneels, Beegle, and Dillon (2017) find that proxying does not affect estimates of returns to education. Another related paper in this literature is by Kilic et al. (2022), that also intersects with the recall literature on labor measurement. They use two nationally representative surveys in Malawi conducted in the same year but following different research protocols. They find underreporting of certain labor activities in the survey using “business as usual,” that allowed for proxy reporting and presence of other household members, relative to the other survey that required own response and privacy during the interview. Contrasting their setup, with multiple differences between the “business as usual” and the more stringent protocols, our study uses experimental variation designed to isolate a single mechanism, recall windows. Our heterogeneous results, showing greater long recall loss for

> 4 Das and Sanchez-Paramo (2012) find that one-third of acute illness is unreported when comparing one month to one week recall periods. Other notable examples showing high sensitivity of consumption to recall windows include Beegle et al. (2012b), De Weerdt et al. (2016), Backiny-Yetna et al. (2017), and Di Maio and Fiala (2019). Beegle et al. (2012a) find little evidence of distortions from longer recall lags in recording agricultural inputs and production.
proxied individuals, are consistent with Kilic et al. as they compare the two survey methods with both annual and weekly recall.

Finally, the findings in our paper have implications for a range of labor linked papers, especially those in rural settings that rely on similarly constructed data.<sup>5</sup> Large, primarily descriptive literatures currently exist on rural labor, gender and age differences in labor contributions, and rural income diversification. Our results suggest that all these estimates could be meaningfully impacted by reliance on long recall-based data and further distorted by the use of proxy reporting.

We discuss our survey experiment and data in Section 2 and empirical strategy in Section 3. Results are presented in Section 4, followed by a brief discussion in Section 5, and conclusion in Section 6.

# **2. Data**

Our survey experiment is built on a sample of households included in a separate study, the MwAPATA Instituteʼs Malawi Rural Agricultural Livelihood Survey (MRALS), conducted inperson in the fourth quarter of 2019. Their original sample is representative of farm households in rural areas of eight selected districts and used a multitopic household survey to collect information on demographics, health, socio-economic status, time use, and agricultural production from a sample of 3,259 households (Muyanga et al. 2020).<sup>6</sup>

# _2.1 Sample and survey structure_

Our analysis uses a sample of 701 households for whom a complete set of four quarterly phoneinterviews were successfully completed. Starting with the MRALS sample of 3,259 households only 2,435 households had contact phone numbers and were therefore eligible for inclusion in our phone-based survey. Stratifying by region, 1,505 eligible households were randomly selected to be included in our study. 1,020 of these households were successfully contacted and included in Round 1 and ultimately, 701 households were successfully reached for the full set of interviews.

> 5 See Dzanku (2020), Abay et al. (2019), Asfaw et al. (2019), Yeboah and Jayne (2018), Imai et al. (2015), Himanshu et al. (2013), Djurfeldt (2013), Haggblade et al. (2010), Ellis (1998), and Ellis and Freeman (2004) for some recent examples.

> 6 The selected districts include two in the Northern Region (Rumphi and Mzimba), four in the Central Region (Lilongwe Rural, Dowa, Kasungu, and Mchinji), and two in the Southern Region (Neno and Blantyre Rural).
Interviews were conducted by phone and the primary respondent was the respondent from the initial MRALS study, typically the household head. The labor module asked this respondent about their own labor participation and additionally to report the labor activities of up to two other adult household members ages 18 to 65. These adults were pre-selected randomly in households with more than three adults. While survey protocols allowed interviews to be done with other household members if the respondent was unavailable, in practice this happened very infrequently.<sup>7</sup> Column 1 of Table 1 shows descriptive statistics of study participants who were successfully reached for the full set of quarterly interviews and who constitute the sample used in the empirical analysis. Household level characteristics, those of the designated respondent, and those of proxied respondents are shown in panels A, B, and C, respectively. Panel A shows that our final sample contains 701 households, with an average of just over five members split evenly between children and adults. In Panel B we see that 75 percent of primary respondents are men and their average age is 44. Sixty-nine percent are in a monogamous marriage while 10% are in polygamous marriages with 6.6 years of education on average. Panel C shows the characteristics of the other household members whose data was collected in the survey. These household members are younger, 30years-old on average, and 76% are female with similar levels of education.<sup>8</sup> Appendix Table 1 shows these summary statics from the analysis sample with those targeted from the sampling of the initial MRALS sample. They are nearly indistinguishable.

Households were contacted approximately once per quarter over the course of a year. While the surveys were intended to be spaced evenly, three months between surveys, in practice logistical issues affected this timing and led to some variation in the actual interval between interviews. Dates and timing of each survey wave are illustrated in Figure 1. These quarterly surveys were designed to be brief, averaging under 20 minutes, focused on labor activities of adult household members. The labor module followed a similar structure to that used in many household surveys where respondents are asked about their primary work activity, with a recall period of the preceding 90 days. Follow-up questions capture further details about that activity in particular, which months they spent doing it, and how many days and hours they spent on it in a typical month that they were doing it (in that recall window). After reporting that activity, they were asked if

> 7 This occurred in 2.3%, 2.0%, 0.9%, and 0.7% instances across rounds 1 to 4, respectively.

> 8 There is slightly higher attrition among this group resulting from a requirement that individuals were living in the household in all four rounds in order to be included in the analysis sample.
there was a secondary activity that they were engaged in over that same 90-day window and, if so, to provide similar follow-up information.<sup>9</sup> After completing this, the respondent was then asked the same sequence of questions about up to two other adult household members (if available). The interviewers asked to speak to the same primary household respondent in each follow-up round of the survey and asked them about the labor activities of the same family members as those identified and included in the first interview.

In the final interview, respondents were additionally asked to report primary and secondary activities over the past 12 months. Aside from the reference window, the follow-up questions on the activity details were held the same, with the list of months spanning the whole year instead of the last three months. If the sequence of these questions affects respondents’ responses, possibly through either framing effects or fatigue, as has been shown in other studies (e.g., Ambler et al. 2021; Jeong et al. 2023), this could bias the resulting data when using both measures for all individuals in the same order. To address this concern, we therefore randomized the order of the annual recall questions to be either just before or just after the 90-day recall for all family members. In the analysis, we use the reported labor participation of individuals using the recall window which they were asked first, to avoid the possibility of order-induced bias. The analysis then leverages this randomization to test the effect of long versus short recall windows.

Columns 3 and 4 of Table 1 show summary statistics and baseline balance between individuals in households who were randomly assigned to have the short or the long recall questions asked first, respectively. The final column shows the p-value of a test for difference between the two groups. We do not find any statistically significant differences between our two randomly assigned groups.

# _2.2 Outcome variables_

Our analysis focuses on four primary labor supply measures. We examine two extensive margin measures: whether or not the person had any work activities reported about them at all and the number of unique activities they worked; and two intensive margin measures: the number of months worked and the number of hours worked.

> 9 Respondents were then asked about their primary and secondary activities over the preceding week and, if distinct from the 90-day activities, details about those activities. Across all rounds, new jobs were reported in the 7-day category fewer than ten times.
We use the 90-day recall periods to construct quarterly estimates of each of these outcomes. For the intensive margin outcomes, we use the survey date and the variables indicating which months the indicated person worked to calculate the percentage of the month that is applicable for each individual. This is then used to calculate the months and hours worked during that 90-day period.<sup>10</sup> Using the 12-month recall data we create similar quarterly measures to match the dates of the shorter recall.<sup>11</sup> Figure 1 provides a timeline of the four rounds of the surveys, showing the survey dates, the months covered in each recall period, along with the agricultural calendar for maize, the main staple crop in Malawi. We note that there was a delay between the third and fourth (final) survey. This has the effect that some of the time covered in the 90-day recall asked in the Round 1 interview is not well covered by the 12-month recall questions asked at the endline. As such, we observe very low levels of work in Quarter 1 as measured in the 12-month recall data. In particular, we were concerned that including all four rounds of short recall from the four surveys would result in artificially low estimates of work in Quarter 1 for the long recall group. Additionally, these delays resulted in capturing labor data for May and June—the harvest months for the primary agricultural season, in both Round 1 and Round 4.

To address both of these issues, we exclude Quarter 1 from our main analysis and use quarters 2- 4 to improve the comparability of the short and long recall periods. As seen in Figure 1, these three quarters cover the main components of one maize growing season, although there is a small gap between the final two interviews and some seasonal activities just after harvest time may be undercounted and missed in our estimates. This would likely lead to an understatement of long recall losses. We show robustness (and strengthening) of our main results using all four interviews in Appendix Table 1.

To identify the number of jobs done by individuals over the full study period, we linked reported activities across survey rounds, so that if a respondent reported working the same job in all four

> 10 This is necessary because for a 90-day recall period, there are 4 calendar months in which the respondent may have worked. To estimate the months worked or hours worked, we must calculate the relevant fraction of the first and fourth month options.

> 11 We calculate these quarterly measures by using the question in the 12-month recall module that asks respondents to indicate which months they worked in each job, and then use the survey dates from the quarterly interviews to reflect the same time periods covered by the short recall questions. We use the day of survey for each household, so the period covered varies slightly by household based on the date of interview.
rounds it is only counted as one unique job.<sup>12</sup> Finally, we sum the quarterly estimates to create aggregate measures comparable to a 9-month recall. The main analysis compares the aggregated estimates collected by quarterly or long recall.

In comparing labor aggregates using the short and long recall windows, two features of the data should be acknowledged. In the short recall estimates, respondents were able to list different jobs in each round so that, if they reported a different primary and secondary activity in each round, they could have up to eight unique activities.<sup>13</sup> Consistent with common survey practices, the long recall questions only allow for a maximum of two work activities. The opportunity to report a greater diversity of work activities may be an advantage of the short recall estimates. In practice, only 2% of individuals reported to have participated in more than two unique activities based on the short recall data and only 25% of individuals reported to have done two unique tasks in the long recall data (the value at which their responses would be censored). While this mechanical difference in the range of possible reported values may influence the analyses on a handful of individuals, the low incidence of cases in which they would be binding makes it unlikely to be driving the results. We return to this later in our discussion of results.

A second difference is that when calculating hours worked, we rely on a series of questions that ask respondents which months were worked in that time frame, the number of days in a typical month, and then the number of hours in a typical day. In the short recall responses, the number of days and hours apply to those months worked in that quarter, for long recall those values are applied to all months worked throughout the year. The short recall responses therefore allow for greater variability in recorded labor supply within periods worked for the same activity. We also return to this later in the discussion but generally consider the greater flexibility in reported labor intensity across the year using the short recall window to be an important benefit of higher frequency interviews where differences in intensity are missed when relying on annual recall.

The main 9-month aggregate outcome variables described here are summarized in Table 2, separately by short and long recall group (and anticipating the first, main result). Ninety-one

> 12 This was done by preloading the activities reported in earlier survey rounds and asking respondents to indicate when they were referencing the same activity as earlier.

> 13 Unique activities are distinct both by type of employment (e.g., self, wage, etc.) and the employer. For example, if a person reported wage employment with entity A and entity B, these would be considered as two different activities.
percent of those in the short recall group report working at all, in an average of 1.19 activities in 3.6 months for 360 total hours. With the exception of total hours worked these measures are all higher for short recall compared to long recall. Ninety percent of subjects in the short recall group report working on the household farm or in other household agricultural businesses. Much smaller numbers report working in a non-agricultural family business (9.4%) or as wage employees (13.7%). Like the overall measures, these numbers are all lower in the long recall group. Overall, this sample is heavily involved in household agricultural activities, with limited participation in other activities. Non-farm activities, when reported, are more likely to be secondary activities.

# **3. Empirical Approach**

In the final interview, we randomized the order of the questions in the labor module so that some households used the long (annual) recall window first while others used the short (90-day) recall first. For those who received the short recall questions first, we use labor measures based exclusively on their quarterly survey responses, and for those who received the long recall first, we use measures based exclusively on the annual recall responses.<sup>14</sup> Our main analysis leverages this randomization to test for differences between these alternative recall windows using the following main empirical specification: L L L L L L 0 1L L L L L ii ii ii ii The coefficient of interest is 1, the difference in average reported labor measures between = ββ + ββ + XX + εε individuals using long and short recall. is a set of gender by age group by reporting status (proxy ββ ii or self) fixed effects. Age groups are defined as individuals under 25, 25-34, 35-49, and 50 and XX above, approximately quartiles of the study sample. Robust standard errors are clustered at the household level, the level of the treatment. In additional analyses we test for heterogeneity of these differences by proxy status, gender, and age. In these analyses we use fully saturated regression models to test each group’s differences against a null hypothesis of no effect and report p-values for differences across groups.

# **4. Results**

> 14 As discussed in Section 2, we have taken this approach of only using the data from the module that was administered first in the final round to avoid concerns over the potential influence of asking 90-day recall questions just before annual recall questions (or vice versa). Appendix Table 2 shows that this ordering indeed has a substantial impact, particularly on the long recall reported values.
# _4.1 Main outcomes_

Table 3, Panel A, presents the main results. In columns 1 and 2 we examine whether the person worked at all and the total number of unique jobs reported. We find that long recall led to large reductions in both of these measures. Long recall reduces any reported work participation by 20 percentage points relative to short recall. Just 8% of individuals report no work whatsoever in the short recall, so an increase by 20 percentage points constitutes an enormous difference in reported labor force participation. Long recall also reduces the number of unique jobs reported by 0.3, which is 24% of the short recall mean. On the intensive margin of labor supply, column 3 shows a similar pattern for the number of months worked. Long recall reduces the reported number of months worked by 0.7 relative to short recall, 20% of the short recall mean. However, when considering hours worked, in column 4, there is only a small negative coefficient that is not statistically different from zero.

In columns 5, 6, and 7 of Table 3, Panel A, we consider the types of work reported for each individual. We report the impact of long recall on having worked on the household farm or in an agricultural-related home business, participation in a non-agricultural home business, or having done wage labor of any kind. As expected in a rural sample, agricultural work is the most common with 89% of individuals reporting participation in the short recall group. Long recall reduces this rate by 22%. These relative losses are comparable to those for having worked at all. Looking at non-agricultural household business and wage work, long recall reduces reported participation by six and seven percentage points, respectively, from short recall bases of 9 and 14 percent. These translate into very large proportional effects of 65% and 51%, suggesting disproportionate losses in reported non-agricultural labor contributions when using long recall that may reflect important sources of income diversification in this rural sample.

Given such strong effects on the other measures, the lack of detectable effect on the number of hours worked is surprising. One possibility is that respondents report hours worked in high intensity seasons instead of values that correspond more closely to a true mean across all months in a given range. This would mean that while reported hours worked vary between relatively high and low seasons in short recall, the maximal levels of labor intensity end up getting applied to all months worked across the full year in the long recall, generating an inflated estimate of hours even when months and activities are held constant. This pattern of reporting is similar to behaviors
shown in Arthi et al. (2018). It is also consistent with the reporting in our data. For those participating, the average hours per month worked under short recall (averaged across rounds) is 96.4 for primary jobs and 83.4 for secondary jobs. The hours per month for the long recall are notably higher, 111.9 for primary jobs and 115.7 for secondary jobs.

# _4.2 Heterogeneity by self and proxy reporting_

Next, we examine heterogeneous impacts of long recall by proxy/self-reporting, gender, and age. Because a respondent is thought to have more complete knowledge of their own labor contributions than those of other household members, use of long recall when reporting on behalf of others could be more taxing and more susceptible to omission. We present these results in Table 3, Panel B, estimating effects by proxy status using a fully saturated regression model, such that the reported interaction is the long recall effect for each group compared to no effect. Beneath each column, we also report the means in the short-recall group separately for self and proxy reports, the scaled differences of the treatment effects, and the p-value for the difference between the two groups. In considering these patterns we emphasize that a household’s primary respondent, and therefore proxy or self-reporting status, was not randomly assigned and therefore we cannot make causal claims of the effects of proxying itself on reported measures.<sup>15</sup> However, we can show how absolute and relative gaps between long and short recall differ by proxy status in our sample.

Across the first four outcomes, losses from long recall are larger for proxied individuals, in both absolute and relative (to their short recall mean) terms, although we lack sufficient precision to say whether these differences are statistically significant for the two intensive margin measures. Long recall loss for working at all among those who self-report is 5 percentage points, or 5% considering a short recall mean of almost 100%. The effect among those proxied is large: 42 percentage points or 54% of the short-recall mean for proxied individuals. These large differences between self and proxy reports may be due to increased recall bias in proxy reports or to the fact that primary respondents are engaged in different types of labor activities that may be less vulnerable to omission. When considering total activities and months worked (columns 2 and 3), the proportional effect is larger than for working at all (15% compared to 5%) suggesting that recall bias may differentially affect the reporting of a second job or the specific months worked for those

> 15 The survey respondents are almost exclusively household heads and, as reported in Table 1, are overwhelmingly male and significantly older than the proxied respondents.
who are self-reporting. For these measures, the effect of long recall continues to be larger among the proxied respondents, both in absolute magnitude and percent effect. However, the difference is not statistically significant for months worked. As in the overall sample, there is no impact on hours worked, even while the point estimates are consistent with proxy reported labor being more vulnerable to long recall loss.<sup>16</sup>

# _4.3 Heterogeneity by gender and age_

Next, we examine differences in long recall loss by gender and age within respondent type (self or proxy). We begin with heterogeneity by gender, focusing on the impacts of long recall on total activities and number of months worked in Table 4, with worked at all and hours worked reported in Appendix Table 3. For each outcome we estimate three specifications testing for different types of heterogeneity: (1) respondents (self-reports) only, considering heterogeneity by their own gender, (2) proxied individuals only, considering the gender of the household’s respondent, and (3) proxied individuals only, considering heterogeneity by their own gender. We do not observe any meaningful differences in long recall loss by respondent gender among self-reported measures in columns 1 and 4 or proxied measures in columns 2 and 5. Losses among proxied women appear to be larger in absolute and relative terms, especially for number of months worked in column 6, but the estimates lack precision to distinguish these differences from statistical noise.

Finally, we perform a similar analysis on effects by age group in Table 5 and Appendix Table 4, replicating the specifications and table structure from the gender analysis. While tests for differences between all four age groups do not always show statistical significance, it appears that older respondents have higher levels of long recall loss when reporting their own labor activities. Although column 2 does not show clear patterns, column 5 suggests that this pattern may be reversed when reporting about other household members, with older respondents exhibiting less long recall loss than younger respondents. Finally, younger household members appear to be more affected by long recall loss in both absolute and relative magnitude with significant differences among those under 25 or 25-34 when tested against people 50 and above.

> 16 Regarding types of activities in columns 5-7 the patterns for farm work are similar to working at all. For nonagricultural businesses and wage work we observe larger absolute effects for those who are self-reporting; the proportional effect for non-agricultural businesses is larger for respondents and the effect on wage work is larger for proxies. In the latter case however, we cannot reject that the two effects are equal.
# _4.4 Time pattern of results_

To better understand the dynamics and mechanics of these results, we show results for the four main outcomes by quarter, again focusing on quarters 2, 3, and 4 in Figure 2. Consistent with our discussion in section 2, we omit quarter 1 as the long recall window did not sufficiently overlap with the first short-run recall to afford comparable reference periods. Each panel shows the regression adjusted means by quarter, with a 95% confidence interval on the difference between the two recall groups.

Across all four outcomes, we see that long recall labor measures are furthest below the short run measures in the second quarter, the quarter furthest from the time of the final interview, and the difference is highly significant, showing proportionate reductions of approximately 40%. Reported differences in labor participation for any work, number of activities, and number of months worked grow smaller as the amount of time since the endline is reduced in quarters 3 and 4. However, the intensive measure of hours worked shows a sudden re-widening in the final quarter. And for both the number of months and hours worked, the long recall responses in quarter 4 are significantly greater than those based on the short recall-based responses.

The patterns shown in these results are puzzling given that major activities in the long recall should not be missed by the major activities that take place in 90-day segments within that range. One behavior that could lead to increasing labor measures over time in the long recall is if respondents have a tendency to “pull forward” the attributed months of a given activity from the months they report having started it up towards the present day at the time of the endline (in quarter 4). If, as discussed in section 4.1 respondents then apply their highest levels of participation for these activities, this could result in especially dramatic exaggerations in quarter 4. Despite this caveat, we generally observe meaningfully lower reported labor measurements when using long recall, with effects that increase along with time farther from the endline.

# **5. Discussion and Interpretation**

While we have discussed these gaps between long and short recall measurements as “long recall losses”, a few additional points deserve consideration in interpreting these differences as losses. First, the majority of current research suggests that reported data is less accurate with longer recall windows, however even the shorter, three-month recall window that we use as our benchmark may
be missing meaningful activities that could be captured using even shorter interview intervals and recall periods. To the extent that our short recall responses are themselves suffering from recall loss, our estimated long recall losses will underestimate the full extent of missed labor contributions.

Second, telescoping, the crowding in of actions that actually occurred just outside of the intended recall window, could lead to exaggeration of reported behaviors (Abate et al. 2022). While this phenomenon can affect responses using either short or long recall, short recall estimates could be especially impacted when aggregating across multiple survey rounds to characterize longer windows of time. However, the use of quarterly interviews may have also served as natural (if not explicit) time markers, bounding the time frame of their responses, which has been shown in other settings to limit the influence of telescoping (Abate et al. 2022). Further, asking individuals to specify specific months of participation in activities may further limit risks of these effects.

A final consideration is that the act of responding to additional interviews, as used to collect the short recall data, might itself impact peoples’ survey responses or affect attrition (Arthi et al. 2018; Zwane et al. 2011). We note that because everyone in our analysis sample was interviewed the same number of times, this does not threaten the internal validity of our results, though it could influence whether we think we would get the same long-recall responses in the absence of the earlier interviews. The setup of our study prevents us from directly testing this although it could be an important question for future research.

Despite these caveats, this work suggests that there are advantages to the three-month windows, although future research could evaluate their performance against even shorter recall periods to further inform these tradeoffs along with their vulnerability to telescoping or distortions induced by repeated interviews. Ultimately, researchers will need to carefully consider their context and the objectives of their research. They must weigh the value of more precise shorter recall measurements and the improved ability to capture seasonal, secondary, and informal labor contributions against survey costs and the potential for respondent fatigue.

# **6. Conclusion**

The results presented in this paper suggest that using an annual recall window to measure seasonal variability in employment can lead to considerable losses relative to the use of a shorter recall
window in a set of quarterly interviews. Secondary, non-agricultural sources of income are disproportionately impacted. The extent of the losses that we observe in our analysis are heavily influenced by whether an individual’s labor contribution is being self-reported or reported by the household’s primary respondent. Labor contributions of youth are more likely to be omitted when using longer recall windows and relying on proxy reports. Because women and youth are less likely to be household heads and therefore more likely to be proxied in household surveys, the use of long recall periods may especially undercount their labor contributions. The time pattern of results is suggestive of more underreporting the further is the time from the survey time frame, coupled with possible overestimation of hours worked.

Given these losses, use of long recall windows can fundamentally affect our understanding of time patterns in peoples’ labor contribution, even when respondents are asked to disaggregate their work over the course of that period. This is not a principal goal of many multi-topic surveys that are primarily focused on the number of months worked and with even less emphasis on which ones. However, as policy makers increasingly demand data driven insights, continuing to refine and improve our methods of data collection is becoming more important. Understanding the seasonality, intensity, and diversity of labor is central to rural development planning. As this interest in the intensity, diversity, and seasonality of work continues to grow, an accompanying understanding of how to accurately measure labor contributions is becoming increasingly important. This objective cannot be separated from a need to understand how survey design and measurement choices, such as recall windows and proxy reporting, impact the resulting data, and finding ways to reduce their influence and potential biases.