metadata: authors: 0: author: Koroknay-Palicz,Tricia
1: author: Montalvao,Joao
count: Africa
docna: 0: docna: Sex, Lies, and Surveys : The Role of
            Interviewer Characteristics
docty: Policy Research Working Paper
theme: Gender
majtheme: Social dev/gender/inclusion
subtopic: Gender and Development,Education for Development (superceded),Educational Populations,Education For All,Educational Sciences,Inequality
teratopic: Education,Poverty Reduction,Gender
abstracts: cdata!: This paper examines how easily
            observable interviewer characteristics, such as gender and
            physical attractiveness, and more difficult to observe
            characteristics, such as attitudes and beliefs, affect
            adolescent girls' disclosure of sexual behavior during
            a baseline survey for an adolescent girls program in
            Liberia. The study finds that girls are more likely to
            report sexual activity to better-looking interviewers, and
            less likely to do so to interviewers holding more
            discriminatory gender attitudes and greater expectations
            about the program. The study finds no evidence that the
            gender of the interviewer matters.
repnme: repnme: Sex, Lies, and Surveys : The Role of
            Interviewer Characteristics
display_title: Sex, Lies, and Surveys : The Role of
            Interviewer Characteristics
listing_relative_url: /research/2019/02/30816843/sex-lies-surveys-role-interviewer-characteristics
topicv3: Education, Skills Development and Labor
            Market m1326135 1484,Poverty m1326687 695,Education m1327048 470,Soft Skills m1327287 2137,Gender m1327621 615
publishtoextweb_dt: 2019-02-07T00:00:00Z
historic_topic: Education,Poverty Reduction,Gender
no_of_pages: 9
author_keys: 0: author_key: 1222184
model_extractions: WPS8732 Policy Research Working Paper 8732 Sex, Lies, and Surveys The Role of Interviewer Characteristics _Tricia Koroknay-Palicz Joao Montalvao_ Africa Region Gender Innovation Lab February 2019
Policy Research Working Paper 8732

# **Abstract**

This paper examines how easily observable interviewer characteristics, such as gender and physical attractiveness, and more difficult to observe characteristics, such as attitudes and beliefs, affect adolescent girls’ disclosure of sexual behavior during a baseline survey for an adolescent girls program in Liberia. The study finds that girls are more likely to report sexual activity to better-looking interviewers, and less likely to do so to interviewers holding more discriminatory gender attitudes and greater expectations about .

This paper is a product of the Gender Innovation Lab, Africa Region. It is part of a larger effort by the World Bank to provide open access to its research and make a contribution to development policy discussions around the world. Policy Research Working Papers are also posted on the Web at http://www.worldbank.org/research. The authors may be contacted at jmontalvao@worldbank.org.

_The Policy Research Working Paper Series disseminates the findings of work in progress to encourage the exchange of ideas about development issues. An objective of the series is to get the findings out quickly, even if the presentations are less than fully polished. The papers carry the names of the authors and should be cited accordingly. The findings, interpretations, and conclusions expressed in this paper are entirely those of the authors. They do not necessarily represent the views of the International Bank for Reconstruction and Development/World Bank and its affiliated organizations, or those of the Executive Directors of the World Bank or the governments they represent._ Produced by the Research Support Team
# **Sex, Lies, and Surveys: The Role of Interviewer Characteristics**<sup>*****</sup>

Tricia Koroknay‐Palicz Joao Montalvao<sup>†</sup>

**Keywords:** Sexual behavior, gender, interviewer effects, beauty, attitudes.

**JEL codes:** C81, C83, I12, J13

> * We thank all those at Innovations for Poverty Action for collaborative efforts in this project. We thank Virginia Ceretti, Shubha Chakravarty, Dackermue Dolo, Cecile Fanton D’Andon, Juliette Seban, and Malay Taylor for project support. We have benefited from comments from Julian Jamison. This paper is a product of the Africa Gender Innovation Lab. We gratefully acknowledge funding from the World Bank Umbrella Fund for Gender Equality. The study protocol was approved by the University of Liberia – Pacific Institute for Research and Evaluation Institutional Review Board.  The views presented in this paper are those of the authors and do not represent those of the World Bank or its member countries. All errors remain our own.

> † Both authors are affiliated with the World Bank Africa Region Gender Innovation Lab. Emails: tgonwa@worldbank.org, jmontalvao@worldbank.org.
# **1. Introduction**

Early sexual initiation puts adolescent girls at risk for early childbearing and marriage. To test the impact of policy interventions, researchers often rely on self‐reported sexual behavior data obtained from face‐ to‐face surveys [e.g. Duflo _et al._ 2015]. However, survey methodologists have long been concerned with respondents’ tendency to misreport sensitive behaviors to avoid embarrassment or possible repercussions of disclosing the truth [Tourangeau and Yan 2007].

To assess those threats and adjust responses accordingly, respondents must form impressions of and make inferences about the preferences, values, and intentions of the interviewer. Most research on the role of interviewer characteristics on misreporting of sensitive behaviors focuses on Western adult respondents and observable interviewer characteristics, such as race and gender [West and Blom 2016]. In contrast, we use data from adolescent girls in Liberia and examine how self‐reports of sexual behaviors vary by both observable (gender, physical attractiveness) and seemingly unobservable (attitudes, beliefs) interviewer characteristics.

Our results highlight the importance of physical appearance and nonverbal communication in the social cognition processes that take place in a survey. In particular, interviewer’s physical attractiveness could evoke stereotypes among survey respondents, such as the belief that better‐looking individuals are sexually warmer or more trustworthy as long emphasized by the social psychology literature [e.g. Langlois _et al._ 2000]. Whether such stereotypes induce girls to report more or less truthfully is ambiguous. For example, if girls expect better‐looking interviewers to be sexually warmer, they may over‐report sexual activity to impress them. Or if girls see them as more trustworthy, they might feel more comfortable to disclose the truth to them.

Perhaps more surprising is the evidence that girls’ responses are also affected by seemingly unobservable interviewer attitudes, given the structured and scripted nature of the interview. This finding suggests that interviewers signal (possibly unconsciously) their views through nonverbal cues, such as subtle changes in gaze or intonation. Girls appear to pick up those cues and adjust their responses accordingly. Girls might be less likely to disclose sexual activity to interviewers who they perceive to judge more harshly such 2
behavior to avoid embarrassment, or if they believe the enumerator has a role in influencing their selection into the program and they fear that telling the truth will harm their chances of being selected.<sup>1</sup>

# **2. Data**

# **2.1 Girls’ Sexual and Romantic Activity**

This study is embedded within a baseline survey for an impact evaluation of the Sisters of Success (SOS) Program, in which mentors and girls’ groups delivered life skills, including sex education, to adolescent girls aged 12‐15. The baseline survey took place from October 2013 through January 2014, with girls who registered for the SOS program. This paper uses data from the 1,264 girls aged 14‐15 during baseline, as incidence of reported sexual activity among the younger girls is too low for statistical inference. We further drop 46 observations for whom some data used in this analysis are missing, for a final working sample of 1,210 girls. Among these, 17% reported having ever had sex, 10% having had sex in the past thirty days, 20% having ever had a boyfriend, and 15% currently having a boyfriend.<sup>2</sup>

# **2.2 Interviewers’ Characteristics**

We examine how girls’ self‐reports vary with the characteristics of the 40 enumerators who interviewed them. We use data from a survey self‐administered by the interviewers in May 2014 to construct measures of attitudes towards gender, attitudes towards the SOS program, and ability using selected questions that predict girls’ responses. We also rated interviewers’ physical attractiveness.

# **2.2.1 Physical Attractiveness**

To measure physical attractiveness, we follow a similar procedure to that used in the economics literature on beauty [e.g. Hammermesh and Biddle 1994]. Each interviewer was rated independently on a scale from 1 (“Very unattractive”) to 10 (“Very attractive”) by a male‐female pair of young Liberian adults, hired as research assistants for this study. The raters have similar standards: the mean enumerator physical attractiveness is 5.6 (s.d.=1.6) for the male rater, 5.2 (s.d.=1.5) for the female rater, and the Cronbach’s

> 1 One cannot also rule out the possibility that girls leave wiggle room in their verbal responses, and the enumerators with different attitudes code it differently.

> 2 In this paper we focus on these extensive margins of girls’ sexual and romantic behaviors. We find little variation on the corresponding intensive margins (e.g. number of sexual partners the girl ever had).

3
alpha of their ratings is 0.81.<sup>3</sup> To create a single index, we employ a double standardization method. First, we standardize both ratings to have mean zero and standard deviation one. Second, we standardize the average of the two standardized ratings to have mean zero and standard deviation one. A higher index score means greater physical attractiveness.

# **2.2.2 Attitudes**

We construct a gender attitudes score by aggregating interviewers’ level of agreement with four statements: “ _Females are not as good as males in school_ ”; “ _Girls can make as good leaders as boys_ ”; “ _If a woman differs with her husband, she must accept his opinion_ ”; and “ _In some circumstances, it is justifiable for a man to beat his wife_ ”. Possible responses were on a scale from 1 (“ _Really agree_ ”) to 4 (“ _Really do not agree_ ”). We recode negative items such that higher values mean more gender equitable views. We standardize the score to have mean zero and standard deviation one.

We measure interviewer attitudes towards the SOS program by asking: “ _How big of a difference do you think a program, like the SOS program, which matches girls to mentors and creates girls’ groups, will make in the lives of the girls who participate in it?_ ”, with possible responses on a scale from 1 (“ _Least_ ”) to 5 (“ _Most_ ”). Among the respondents, 53% selected 5 (“Most”); 27%, 4; and 8%, 3. Given this skewed distribution, we define _mentorship‐enthusiastic_ enumerators as those who selected “Most”.

# **2.2.3 Ability**

To control for interviewer ability, we construct scores for “hard” and “soft” skills. The hard skills score adds two indicators: having prior survey experience (48% have) and a university degree (53% have). The soft skills score combines information on patience and ability to communicate with girls about sex. Patience is measured by asking: “ _On a ladder from 1 to 5, where 1 is ‘not very patient’ and 5 is ‘very patient’, where do you stand?_ ”. We measure communication skills with the question: “ _I feel uncomfortable talking with adolescent girls about their sexual behavior_ ”, with possible responses on a scale from 1 (“ _Really agree_ ”) to 4 (“ _Really disagree_ ”). The score adds two indicators for reporting to be very patient (75% do) and very comfortable talking to girls about sex (65% do).

> 3 A higher Cronbach alpha coefficient means greater concordance among raters. The Cronbach alpha for beauty reported in Hamermesh and Biddle [1998] is 0.75.

4
# **3. Results**

We estimate the following logit model for girl i interviewed by interviewer j :

# y�� �α�βAttractiveness�� �γGender�� �δAtitudes�� �ηX�� �λW�� �ε�� ,

where y is the girl’s dichotomous response of interest. Attractiveness is the measure of interviewer physical attractiveness. Gender is a dummy equal to one if the interviewer is male. Attitudes is a vector of interviewer attitudes towards gender and the SOS program. X is a vector of interviewer controls, including age and ability scores. W is a vector of respondent controls, including age, years of schooling, household size, number of assets owned by the girl’s household, and dummies for household head’s educational attainment. ε�� is a disturbance term that we allow to be clustered by interviewer.

Table 1 reports the results. Panel A only controls for interviewer characteristics. The first row shows that girls are more likely to report sexual and romantic activity to better‐looking interviewers. A one standard deviation increase in interviewer’s physical attractiveness significantly increases the likelihood that girls report having ever had sex by about 3 percentage points (pp), having ever had a boyfriend by 5pp, having had sex in the past thirty days by 3pp, and currently having a boyfriend by 4pp. The second row shows interviewer gender has no significant impact on girls’ responses, which is in line with the literature showing many null results [see West and Blom 2016, for a review].

The third row shows that girls are more likely to disclose sexual and romantic activity to interviewers holding more gender equitable views: a one standard deviation increase in interviewer’s gender attitudes significantly raises the likelihood that girls report having ever had sex, a boyfriend, and currently having a boyfriend by about 3pp. The fourth row shows that girls interviewed by interviewers that are enthusiastic about the program are significantly less likely to report having ever had sex by 8pp, having ever had a boyfriend by 8pp, having had sex in the past thirty days by 6pp, and currently having a boyfriend by 7pp.

The fifth and sixth rows show that interviewer ability also matters. Girls are more likely to disclose sexual and romantic activity to higher ability enumerators. The magnitudes of the coefficients multiplying the soft skills score are substantially larger, and in general more precisely estimated than those multiplying the hard skills score. We find no systematic evidence that the age of the enumerator has an impact on girls’ responses. Panel B shows these findings are robust to additionally controlling for respondent 5
characteristics, which helps mitigate against the concern that our results are potentially biased by the non‐random assignment of interviewers to respondents.

# **4. Discussion**

Our findings illustrate that interviewer characteristics can impact survey data. A methodological implication is that researchers may wish to randomly assign enumerators to respondents, or at a minimum control for interviewer fixed effects in regression analysis of survey data. A substantive implication is that, to the extent that the social interaction during a survey interview echoes social interactions in the real world, our results can offer insights into the psychological mechanisms through which vulnerable adolescent girls perceive others and strategically attempt to manage how they are perceived.

# **References**

- [1] <mark>DUFLO.E, P.DUPAS AND M.KREMER (2015). “Education, HIV, and Early Fertility: Experimental Evidence from Kenya.”</mark> _<mark>American Economic Review</mark>_ <mark>,</mark> _<mark>105</mark>_ <mark>(9): 2757‐97.</mark>

- [2] HAMERMESH.D.S. AND J.E.BIDDLE (1994). “Beauty and the Labor Market.” _American Economic Review_ , _84_ (5): 1174‐94.

- [3] LANGLOIS.J.H., L.KALAKANIS, A.J.RUBENSTEIN, A.LARSON, M.HALLAM AND M.SMOOT (2000). “Maxims or Myths of Beauty? A Meta‐Analytic and Theoretical Review.” _Psychological Bulletin_ , _126_ (3): 390‐423.

- [4] TOURANGEAU.R AND T.YANG (2007). “Sensitive Questions in Surveys.” _Psychological Bulletin_ , _133_ (5): 859‐883.

- [5] WEST.B.T AND A.G.BLOM (2017). “Explaining Interviewer Effects: A Research Synthesis.” _Journal of Survey Statistics and Methodology_ , _5_ (2): 175‐211.

6
# **Table 1: Interviewer Characteristics and Adolescent Girls’ Responses to Questions Related to Sexual and Romantic Behavior**

## **Logit marginal estimates, standard errors clustered by interviewer reported in parentheses**

||**Lifetime B**|**ehavior**<br>|**Current B**<br>|**ehavior**<br>|
|---|---|---|---|---|
||**(1) Ever had sex**|**(2) Ever had**<br>**boyfriend**|**(3) Had sex in past**<br>**30 days**|**(4) Has boyfriend**<br>**currently**|
|**_Mean of dependent variable:_**|.170|.204|.102|.150|
|**_Panel A: Interviewer Controls_**|||||
|**Physical Attractiveness [z‐score]**|.034**|.045**|.028**|.044***|
||(.015)|(.019)|(.014)|(.015)|
|**Male [yes=1]**|.019|.035|‐.001|.023|
||(.024)|(.031)|(.029)|(.026)|
|**Gender Attitudes [z‐score]**|.027**|.035**|.003|.033***|
||(.014)|(.014)|(.013)|(.013)|
|**Program Enthusiastic [yes=1]**|‐.079***|‐.077**|‐.059***|‐.066**|
||(.027)|(.032)|(.020)|(.029)|
|**Soft Skills [0‐2 score]**|.072***|.084***|.067***|.080***|
||(.025)|(.027)|(.020)|(.022)|
|**Hard Skills [0‐2 score]**|.056**|.043|.038*|.061**|
||(.028)|(.029)|(.023)|(.026)|
|**Age**|.001|.004*|‐.001|.003*|
||(.002)|(.002)|(.002)|(.002)|
|**Respondents [interviewers]**|1,210 [40]|1,210 [40]|1,210 [40]|1,210 [40]|
|**_Panel B: Interviewer and Respondent Controls_**|||||
|**Physical Attractiveness [z‐score]**|.032**|.045***|.024**|.042***|
||(.013)|(.016)|(.012)|(.013)|
|**Male [yes=1]**|.018|.036|‐.002|.020|
||(.022)|(.028)|(.023)|(.021)|
|**Gender Attitudes [z‐score]**|.024**|.033***|.001|.029***|
||(.012)|(.012)|(.010)|(.010)|
|**Program Enthusiastic [yes=1]**|‐.054**|‐.055*|‐.041**|‐.048*|
||(.025)|(.029)|(.017)|(.026)|
|**Soft Skills [0‐2 score]**|.070***|.085***|.059***|.075***|
||(.022)|(.024)|(.016)|(.019)|
|**Hard Skills [0‐2 score]**|.063**|.054**|.039**|.064***|
||(.025)|(.027)|(.018)|(.022)|
|**Age**|.000|.003|‐.001|.002|
||(.002)|(.002)|(.002)|(.002)|
|**Respondents [interviewers]**|1,210 [40]|1,210 [40]|1,210 [40]|1,210 [40]|

**Notes:** *** denotes significance at 1%, ** at 5%, and * at 10% level. All columns report logit estimates, where marginal effects are reported in each case. Standard errors are in parentheses, and are clustered by interviewer. Panel A only controls for interviewer characteristics. Panel B also controls for respondent controls, which include girl’s age, years of schooling, household size, number of assets owned by the household, and dummies for household head’s educational attainment.

7