metadata: authors: 0: author: Halim,Daniel Zefanya
1: author: Powers,Elizabeth T.
2: author: Thornton,Rebecca Lynn
count: World
docna: 0: docna: Gender Differences in Economics
            Course-Taking and Majoring : Findings from an RCT
docty: Policy Research Working Paper
subtopic: Gender and Development,Labor Markets,Educational Sciences,Economics of Education,Education For All,Education for Development (superceded),Educational Policy and Planning - Textbook,Educational Populations
teratopic: Social Protections and Labor,Education,Gender
abstracts: cdata!: This paper reports on gender differences
            in responses to a randomized controlled trial that provided
            encouragement and information nudges to take subsequent
            economics courses and major in the subject for students
            enrolled in large introductory economics classes at a large
            elite public university. Two treatments combined
            encouragement to major in economics with information on
            either financial or prosocial returns to the major. Men
            receiving either treatment were more likely to take an
            additional economics course, but not to major in economics.
            In contrast, the treatments were not estimated to
            significantly affect women’s course-taking and majoring. Two
            treatment mediators are also examined: expected versus
            actual grade and having a female teaching assistant. There
            were also differing effects of mediators on treatment
            responses for men and women. Women were more nudge-able to
            take another course when they received a
            better-than-expected introductory class grade, and men were
            more nudge-able to take another course when they had a
            female teaching assistant.
repnme: repnme: Gender Differences in Economics
            Course-Taking and Majoring : Findings from an RCT
display_title: Gender Differences in Economics
            Course-Taking and Majoring : Findings from an RCT
listing_relative_url: /research/2021/07/33279080/gender-differences-economics-course-taking-majoring-findings-rct
topicv3: Textbooks m1326802 2129,Education m1327048 470,Economics of Education m1327511 1540,Gender m1327621 615,Jobs m1503931 2670
publishtoextweb_dt: 2021-07-19T00:00:00Z
dois: 10.1596/1813-9450-9732
historic_topic: Social Protections and Labor,Education,Gender
origu: Gender Director (HGNDR)
no_of_pages: 17
author_keys: 0: author_key: 1483795
1: author_key: 684219
model_extractions: Policy Research Working Paper 9732 Gender Differences in Economics Course-Taking and Majoring Findings from an RCT _Daniel Halim Elizabeth T. Powers Rebecca Thornton_ Gender Global Theme July 2021
Policy Research Working Paper 9732

# **Abstract**

This paper reports on gender differences in responses to a randomized controlled trial that provided encouragement and information nudges to take subsequent economics courses and major in the subject for students enrolled in large introductory economics classes at a large elite public university. Two treatments combined encouragement to major in economics with information on either financial or prosocial returns to the major. Men receiving either treatment were more likely to take an additional economics course, but not to major in economics. In contrast, the treatments were not estimated to significantly affect women’s course-taking and majoring. Two treatment mediators are also examined: expected versus actual grade and having a female teaching assistant. There were also differing effects of mediators on treatment responses for men and women. Women were more nudge-able to take another course when they received a better-than-expected introductory class grade, and men were more nudge-able to take another course when they had a female teaching assistant.

This paper is a product of the Gender Global Theme. It is part of a larger effort by the World Bank to provide open access to its research and make a contribution to development policy discussions around the world. Policy Research Working Papers are also posted on the Web at http://www.worldbank.org/prwp. The authors may be contacted at dhalim@worldbank.org.

_The Policy Research Working Paper Series disseminates the findings of work in progress to encourage the exchange of ideas about development issues. An objective of the series is to get the findings out quickly, even if the presentations are less than fully polished. The papers carry the names of the authors and should be cited accordingly. The findings, interpretations, and conclusions expressed in this paper are entirely those of the authors. They do not necessarily represent the views of the International Bank for Reconstruction and Development/World Bank and its affiliated organizations, or those of the Executive Directors of the World Bank or the governments they represent._ Produced by the Research Support Team
# Gender Differences in Economics Course-Taking and Majoring: Findings from an RCT

Daniel Halim<sup>a</sup> , Elizabeth T. Powers<sup>b1</sup> , Rebecca Thornton<sup>c</sup> a World Bank, Washington, D.C.

b University of Illinois at Urbana-Champaign, Urbana, Illinois c University of Illinois at Urbana-Champaign, Urbana, Illinois _JEL Classification:_ C93, J16, J7, A22.

_Keywords:_ Gender; Economics; RCT; College Major.

> * Correspondence to: Department of Economics, University of Illinois at Urbana-Champaign, Urbana, Illinois, 61801. _Email address_ : epowers@illinois.edu (E. Powers).

> This research was approved by the Institutional Research Board at the University of Illinois at Urbana-Champaign. We acknowledge funding from the NBER WiE (Women in Economics) program. We thank Melissa Newell for key help implementing early waves of the RCT, and the instructors of the introductory economics courses for their help enrolling students in the study.
# **Introduction**

Gender differences in college major drive occupational segregation and inequality, reduce diversity of thought, and may cause women or men to miss out on personally beneficial knowledge. A large literature has found that post-college financial return is the marginal determinant of men’s major choice. In contrast, myriad additional factors influence women, including perceived ability, grades, intrinsic enjoyment of the subject, and work-life balance goals (Altonji, Blum, Meghir 2012; Zafar, 2013; Wiswall & Zafar, 2016). While men may respond to information about financial returns, women may be more likely to enter a traditionally male major if it is more suited to their intellectual interests, aptitudes, or career and life goals (Bansak & Starr, 2010; Bertrand, 2018).

We study the effect of providing information about labor market or pro-social returns to undergraduate students enrolled in introductory economics courses at a large public university. Economics is an interesting and relevant context for the study. It is a financially as well as intrinsically rewarding major, yet women have been a minority of economics majors (less than one-third) for decades (Lundberg and Stern, 2019). Studies investigating program-specific factors that may discourage women from moving beyond introductory classes have examined women’s perception of economics as a “business” topic (Bansak & Starr, 2010), the impact of female instructors and role models (see Blau, Currie, Croson, and Ginther, 2010; Porter & Serra, 2020; Emerson, Goldrick, & Siegfried, 2017; Jensen & Owen, 2001; and Robb & Robb, 1999), discouraging experiences in introductory courses (Jensen & Owen, 2001), gendered instructional materials (Stevenson & Zlotnick, 2018; Jensen & Owen, 2001; Lage & Treglia, 1996), and grade sensitivity (Goldin, 2015; Main & Ost, 2014; Owen, 2010; and Rask & Tiefenthaler, 2008).

A large public university is also an appropriate setting for studying interventions to raise economics course-taking and majoring. Relatively few studies have been carried out at very large public universities. There are more majors to choose among than at smaller institutions, and high student-teacher ratios engender very different pedagogical practices.

Surprisingly few randomized controlled trials (RCTs) have evaluated interventions to retain college women in economics.<sup>2</sup> We implemented an RCT among undergraduate students enrolled in large introductory economics courses at the University of Illinois at Urbana-

> 2 An exception is Porter & Serra (2020) who expose women in principles classes at a small, private liberal arts college to compelling female role models, finding that this simple intervention almost doubles the number of female economics majors.

2
Champaign. Two treatment arms provided encouragement to major in economics. A “prosocial” treatment provided information emphasizing the wide variety of career options and personal benefits associated with the major, while an “earnings” treatment provided information on financial returns. We evaluate the effects of the two treatments on subsequent choices to take another economics course and declaration of the economics major by the end of the student’s junior year using student-level matched administrative data. In addition to examining the overall effects of these treatments, we examine whether treatment effects are mediated by the gap between introductory-course performance and expected grade and having a female teaching assistant (TA).

Our primary aim is to evaluate whether women can be “nudged” into a major with lowcost, theoretically grounded, encouragement/information interventions. In addition to contributing to the college major literature, our work speaks to the broader literature on gender differences in behavior, including whether women are more prosocially motivated than men and gender differences in responsiveness to small nudges and framing.

We find that the average male student receiving either treatment is more likely to take at least one more economics course after the focal course, but there is little evidence of increased majoring. The average woman appears unresponsive to either treatment. We next consider mediating factors suggested by the prior literature, finding that the introductory course grade is an important mediator for women, while TA gender is an important mediator for men. Women with better-than-expected grades can be nudged to take another economics course, but there is no effect on majoring. Men with a female TA in the focal course are more likely to take another economics course in response to treatment, with no effect on majoring.

# **2. Experimental Design and Analysis**

# 2.1 Setting

The study took place at the University of Illinois at Urbana-Champaign, a large elite public university. The economics program is housed in the College of Liberal Arts and Sciences. When students register for courses, they sign up for both a lecture with a faculty instructor and one of many recitation sections led by TAs, who are graduate students in the economics Ph.D. program.

In addition to being a popular major, introductory economics courses often satisfy requirements for students pursuing other majors (most notably business and engineering), and introductory-level economics classes also provide general studies credits, which are required of all 3
undergraduates. Lectures are very large, with dominant economics textbooks comprising the course materials.<sup>3</sup>

# 2.2 Study Enrollment and Survey

Our study enrolled students in introductory economics courses at the beginning of the Fall 2015, Spring 2016, or Fall 2016 semesters. There is no overall (combined micro-macro) “principles” class. Students in our study were enrolled in introductory microeconomics, introductory macroeconomics, or statistics.

At the beginning of the semester, students were offered a small number of extra credit points for responding to an online survey that elicited the student’s preferred major, intensity of their interest in the economics major, and expected performance in the focal introductory course. Consent and participation were nearly universal.

# 2.3 Intervention

At the end of the semester, students who received a B- or better in the focal course and had a cumulative GPA of at least 2.67 were randomly assigned to a control group or to one of two treatments.<sup>4</sup> The two treatments encourage students to major in economics with differing framings. T1 (prosocial) presented encouragement, reassurance on ability, and information on non-earningsfocused benefits of the major, including varied other-regarding careers and personal enrichment. T2 (earnings) presented encouragement, reassurance on ability, and information on earnings returns associated with the major.

Students in both treatment arms received personalized messages from the faculty director of undergraduate studies. Each message had the same opening paragraph encouraging the student to major in economics, citing their performance in the focal course as evidence of their ability. The second paragraph of the letter discussed either earnings or prosocial motivations for majoring in

> 3 The introductory classes were very large and led by a handful of faculty instructors. One instructor, who always taught the same course, was female.

> 4 These cutoffs were chosen in order to avoid encouraging students to major who might not have the ability to do so successfully.

4
economics. Attachments providing supporting evidence for the suggested motivation accompanied the letter.<sup>5</sup> Written communication was delivered in two modes: An official email and a physical letter on university letterhead mailed to a student-provided primary address.

# 2.4 Outcomes

Students are matched to their administrative records to observe the courses they take in subsequent semesters as well as their major and minor declarations. Our last semester of observations is in the Spring semester of 2019.

Treatment may increase retention among those already declared in economics; may cause a switch to economics or the addition of an economics major for students who have declared another major; and undeclared students may respond to treatment by declaring economics. In all cases, course-taking may increase due to an increase in general interest in the subject, including pursuant to changes in major decisions.

# 2.5 Estimation

We estimate the following specification: 1 2 F F 1 ⋅F F F 2 ∗F F F ii ii ii ii ii ii ii ii ii where yy = αα+ ββ is the outcome of student TT1 + ββ TT2 + γγ F _i_ ,<sup>6</sup> + δδ is an indicator of whether student TT1 + δδ TT2 _i_ is randomly assigned + ψψ XX + εε (1) to the Prosocial Treatment Group 1, ii ii indicates that student _i_ is randomly assigned to the yy TT1 Earnings Treatment Group 2, and F F F ii is the female dummy. is a set of individual control TT2 variables, including age when solicited, US citizenship, first-generation college student, race ii ii XX dummies, pre-college academic and math scores indices, declared major when solicited, class standing and term when solicited. Primary effects are estimated using the sample that includes both men and women, while we present the versions of the mediated findings that are estimated separately for men and women. We report robust standard errors in the tables, instead of clustered standard errors because we only have 11 clusters at the term, introductory economic course, and

> 5 Material available at <u>https://rb.gy/mwtgle.</u>

> 6 We also examined treatment effects on the number of economics courses taken over the college career, for those students observed to graduation. None of the treatment coefficients were estimated to differ significantly from zero in this case. These findings are available in the E-appendix.

5
instructor level. We additionally test for Wild Bootstrap and Randomization Inference p-values to complement our findings based on robust standard errors and to address the small number of clusters problem.<sup>7</sup>

# **3. Results**

Sample statistics, basic treatment findings, and mediated findings are presented in this section. We note important differences across the two treatments as well as male-female differences in treatment response. A full set of tables is provided in an electronic appendix.

# 3.1 Sample and Balance

Our primary sample consists of 1,976 students who were freshmen or sophomores during the focal course. Of these, 664 students were assigned to control, 659 to T1 (prosocial treatment), and 653 to T2 (earnings treatment).

Of the students in our sample, 77.9% were enrolled in introductory microeconomics, 12.8% in introductory macroeconomics, and 9.36% in statistics. Of these, 38.3% were enrolled in Fall 2015, 22.4% in Spring 2016, and 39.3% in Fall 2016. We observe 1,885 students (95.4% of the primary sample) to the end of their junior year.

Students are not required to declare a major until the end of their sophomore year. Only 8.9% (176 students) had declared an economics major at the time of the focal course, while 43.6% had declared any major.

Table 1 reports sample statistics, making comparisons across the subsamples of men and women as well as across the control and (two) treatment samples. There are 1,134 men and 842 women in the study. As reported in Table 1, men are more likely to be U.S. citizens, have a higher pre-college math score index, are marginally more likely to have already declared an economics major by the time of solicitation, and express a higher interest in majoring in economics. We failed to find significant differences between men’s and women’s expected grade in the focal course.

> 7 These findings are available in the E-appendix and the results are consistent across specifications.

6
Table 1 indicates that, with a few exceptions, average characteristics are balanced across the treatment arms.<sup>8</sup> Joint tests of the treatments relative to the control group rejected the hypothesis of orthogonality at standard confidence levels, with the exception of the prosocial treatment when p-values were robustly estimated (p=.034). Also, while there are some statistically significant differences between balance and treatment, the normalized differences of each of the 40 variables investigated are smaller than 0.25 (in absolute value), indicating good balance overall (Imbens & Rubin, 2015).<sup>9</sup>

# 3.2 Main Effects

Table 2 presents the main findings for the three samples. The first column presents specifications without controls. Men’s propensity to take another course increases by 7.2-9.5%points with treatment (these are increases of 11-15% above the mean). Treatment is not estimated to have an effect on women’s propensity to take another course. With controls, men’s likelihood of taking another course increases 3.2%-points under the earnings treatment (note that T1 and T2 are not estimated to have differing treatments on the men). The effects of treatments on majoring are not estimated to differ significantly from zero for men. The hypothesis that treatment has no effect on women also cannot be rejected. Findings are similar in magnitude when the sample is narrowed to undeclared students, although the treatment coefficients are no longer estimated to differ significantly from zero at standard confidence levels.

To sum up our first major finding, there is some evidence that men may be nudged by treatment to take another economics course but not to major in economics. There is no evidence that women’s economics course and major choices are affected by treatment.

Table 3 presents mediated treatment effects (findings in the table are all for the unrestricted sample; findings for the other samples are provided in an E-appendix). Treated women with betterthan-expected focal-course performance are nudged to take an additional economics course. The likelihood that a woman takes another course in response to treatment increases by 5.6-5.9%-points with a favorable one-third- grade “surprise”. The hypothesis of treatment effects on women’s

> 8 The hypothesis that freshman students were balanced across groups was rejected at reasonable confidence levels for robust but not bootstrapped p-values. The hypothesis that Black students were not balanced across the groups was rejected at the 95% level or higher for both robust and bootstrapped p-values.

> 9 Our E-appendix contains the complete set of balance statistics.

7
majoring, mediated or not, is rejected. Men’s susceptibility to treatment is invariant with respect to focal course performance.

To sum up our second major finding: The more the focal-course grade exceeds expectations, the more positively women respond to treatment with subsequent course-taking. Grades do not mediate men’s treatment responses.

Next, we examine whether treatment effectiveness is mediated by female-led recitation sections. Findings for women are only suggestive: The effect of having a female TA on women’s course-taking is fairly large in magnitude but imprecisely estimated (in the case of the earnings treatment, the coefficient is marginally significant). On the other hand, men’s course-taking is very amenable to nudges to take more economics from either treatment in the presence of a female TA. As usual, there is no evidence of treatment effects on majoring.

This implies our third major finding: Men with a female TA are more readily nudged into taking another economics course, while there is no mediating effect for women. There is no evidence that having a female TA influences students’ nudge-ability to major.

Finally, we note that across all specifications, the effects of the two treatments were never estimated to differ from each other at standard confidence levels (this was true of the total effect of each treatment in mediated cases as well). Thus our final major finding: Women did not demonstrate a bias towards a pro-social framing, and men did not demonstrate a bias towards a pro-earnings framing.

# **4. Conclusions**

The economics major is an excellent example of the strong and persistent gender segregation in college major choices. We implemented an RCT targeting both earnings and prosocial motives for studying economics. Overall, men could be nudged more readily to pursue further study of economics than women, regardless of treatment.

Notably, we failed to find that women were more affected by a pro-social treatment. This is consistent with past mixed findings on women’s pro-sociality (Croson & Gneezy, 2009) but contradicts Bettinger & Long (2004), who found a pro-business framing seemed to discourage women from economics. In line with Goldin (2015) and Kugler, Tinsley, Ukhaneva (2017), although contrary to Main & Ost (2014), we found that course grades are particularly important 8
for women. Women who received “good” grade surprises were nudge-able to pursuing more economics education. Grades were not a mediator for men.

Having a female TA enhanced treatment effects for men, further encouraging economics course-taking. We did not find evidence of a “role model” effect on women, adding to the mixed findings in the literature. Bettinger & Long (2004) found some evidence that women were less likely to major in economics when their instructor was female. In contrast, an RCT by Porter & Serra (2020) found that exposure to “successful and charismatic [alumna] who majored in economics” had an enormous impact on women majoring in economics at a small liberal arts college. A possible explanation for our findings is that doctoral students are not compelling role models for undergraduates (Bettinger & Long, 2004). Hoffman and Oreopoulous (2009) found some evidence that male college students were less likely to pursue a subject when the introductory instructor was female. Qian and Zafar (2009) found that greater representation of women on faculty attracted more women into engineering but not other subjects, with no systematic responses of men to female faculty representation.

Men and women responded to randomized nudges to take another economics course, at least under some circumstances, but there was no indication that students were nudged into majoring. While we hypothesized that men would be more readily nudged by the earnings treatment and women by the prosocial treatment, tests that treatment effects differed were rejected for all of our specifications, including mediated treatment effects with grade surprises and having a female TA.

Our findings suggest that caution is merited if one’s aim is to increase female representation in the economics field. Treatments appear overall successful at increasing coursetaking, but majoring never increased as a consequence. Further research in a variety of institutional settings is warranted. It may be important to better understand students’ alternatives to the economics major, especially in settings where introductory economics is often required for other majors. In the case of our institution, doing well in several introductory economics courses was required to meet competitive transfer requirements into the Business College. The revelation of a better-than-expected grade, combined with an additional signal of competence in the treatment, may have inspired some women to try for a transfer seat. This would explain why their coursetaking increased but did not lead to majoring. Future research should consider that women with 9
the strongest aptitude for economics may have better alternatives, perhaps in careers where a combination of hard and soft skills are more highly valued.

10