metadata: authors: 0: author: Aramburu,Julian
1: author: Goicoechea,Ana
2: author: Mobarak,Ahmed Mushfiq
count: World
docna: 0: docna: Coding Bootcamps for Female Digital
            Employment : Evidence from an RCT in Argentina and Colombia
docty: Policy Research Working Paper
subtopic: Labor Markets,Educational Sciences,Rural Labor Markets
teratopic: Social Protections and Labor,Rural Development,Education
abstracts: cdata!: This paper evaluates the short-term
            causal effects of a high-quality, intensive, part-time
            computer coding bootcamp for women on skill acquisition and
            employment outcomes. Spots were offered in an oversubscribed
            coding course to a random subset of applicants in Buenos
            Aires, Argentina, and Bogotá, Colombia. The applicants who
            were chosen received a scholarship that covered most of the
            tuition costs of the course. Follow-up data collected
            shortly after the bootcamp ended  indicate that the program
            increased participants’ coding skills, as well as their
            probability of finding a job in technology. Compared with
            other jobs, technology jobs are more likely to offer
            flexible hours and work-from-home arrangements, and generate
            higher job satisfaction. These results are interpreted as an
            improvement in overall job quality. Moreover, the paper
            compares the employment situation of the sample before and
            during the first months of the COVID-19 outbreak. The
            evidence indicates that the program increased participants’
            resilience to a downturn in the labor market.
repnme: repnme: Coding Bootcamps for Female Digital
            Employment : Evidence from an RCT in Argentina and Colombia
display_title: Coding Bootcamps for Female Digital
            Employment : Evidence from an RCT in Argentina and Colombia
listing_relative_url: /research/2021/06/33230991/coding-bootcamps-female-digital-employment-evidence-rct-argentina-colombia
topicv3: Education, Skills Development and Labor
            Market m1326135 1484,Labor Markets m1326488 855,Education m1327048 470,Jobs m1503931 2670,Coronavirus (COVID-19) m1939478 4828
publishtoextweb_dt: 2021-06-29T00:00:00Z
dois: 10.1596/1813-9450-9721
historic_topic: Social Protections and Labor,Rural Development,Education
origu: EFI-FCI-TIC-Firms,Entrep,&Innov. (ETIFE)
no_of_pages: 55
author_keys: 0: author_key: 1792397
1: author_key: 679223
2: author_key: 418988
model_extractions: Policy Research Working Paper 9721 Coding Bootcamps for Female Digital Employment Evidence from an RCT in Argentina and Colombia _Julian Aramburu Ana Goicoechea Ahmed Mushfiq Mobarak_ Finance, Competitiveness and Innovation Global Practice June 2021
Policy Research Working Paper 9721

# **Abstract**

This paper evaluates the short-term causal effects of a high-quality, intensive, part-time computer coding bootcamp for women on skill acquisition and employment outcomes. Spots were offered in an oversubscribed coding course to a random subset of applicants in Buenos Aires, Argentina, and Bogotá, Colombia. The applicants who were chosen received a scholarship that covered most of the tuition costs of the course. Follow-up data collected shortly after the bootcamp ended indicate that the program increased participants’ coding skills, as well as their probability of finding a job in technology. Compared with other jobs, technology jobs are more likely to offer flexible hours and work-from-home arrangements, and generate higher job satisfaction. These results are interpreted as an improvement in overall job quality. Moreover, the paper compares the employment situation of the sample before and during the first months of the COVID-19 outbreak. The evidence indicates that the program increased participants’ resilience to a downturn in the labor market.

This paper is a product of the Finance, Competitiveness and Innovation Global Practice. It is part of a larger effort by the World Bank to provide open access to its research and make a contribution to development policy discussions around the world. Policy Research Working Papers are also posted on the Web at http://www.worldbank.org/prwp. The authors may be contacted at agoicoechea@worldbank.org.

_The Policy Research Working Paper Series disseminates the findings of work in progress to encourage the exchange of ideas about development issues. An objective of the series is to get the findings out quickly, even if the presentations are less than fully polished. The papers carry the names of the authors and should be cited accordingly. The findings, interpretations, and conclusions expressed in this paper are entirely those of the authors. They do not necessarily represent the views of the International Bank for Reconstruction and Development/World Bank and its affiliated organizations, or those of the Executive Directors of the World Bank or the governments they represent._ Produced by the Research Support Team
# Coding Bootcamps for Female Digital Employment Evidence from an RCT in Argentina and Colombia<sup>_∗_</sup>

Julian Aramburu, Ana Goicoechea<sup>_†‡_</sup> , Ahmed Mushfiq Mobarak<sup>§</sup>

**_JEL Codes_ :** C21, I26, J16, J21, J24, J48, O12.

**_Keywords_ :** STEM, Computer Science, Coding, Technology, Labor Market Policy, Bootcamps, Gender, Skills, Employment, COVID-19, Argentina, Colombia.

> _∗_ We thank The World Bank team –Denis Medvedev, Martha Licetti, Andrew Beath, Sebastian Hurtado-Cano, Sarah Lenoble, and Victor Mulas– for their support and contributions throughout the design and implementation of the program. We also thank Digital House and Bogotá Institute of Technology for implementing the bootcamps in Buenos Aires, Argentina and in Bogotá, Colombia, respectively, and Bogotá Chamber of Commerce (CCB), Invest in Bogotá, BICTIA, and Facebook for their partnership to implement the program in both cities. The authors gratefully acknowledge the financial contributions of infoDev Trust Fund donors and the World Bank’s Gender Umbrella Trust Fund. Isabel Miranda, Agustina Suaya, and Inés Taylor provided excellent field coordination assistance. We thank Berk Ozler, Alaka Holla, Nate Barker, Paula Calvo, Zara Contractor, Lucas Finamor and participants in the Yale Labor/Public Economics Prospectus Workshop for helpful comments. Julian Aramburu thanks The MacMillan Center for International and Area Studies at Yale for funding travel to conduct field work. This randomized control trial was registered in the American Economic Association RCT Registry (ID AEARCTR-0003850) and approved by Yale University Institutional Review Board (IRB ID 2000025881). The views expressed herein are those of the authors and do not necessarily reflect the views of The World Bank nor Yale University.

> _†_ Yale University, julian.aramburu@yale.edu.

> _‡_ World Bank, agoicoechea@worldbank.org.

> §Yale University, ahmed.mobarak@yale.edu.
# **1 Introduction**

In recent years, women’s underrepresentation at the occupational and educational level in the science, technology, engineering and math (STEM) sector has gained attention from academics and policy makers alike. In particular, the gender gap in computer science careers is the most salient of all STEM fields (Sax et al., 2017). In Argentina and Colombia, where this study takes place, women represent 57 and 58 percent of all college graduates, but they comprise only 15 and 9 percent of the graduates in the computer science field, respectively (Argentina Ministry of Education, 2015; Colombia Ministry of TIC, 2017). To women put this figure in perspective, this gap in the United States is remarkably similar: represent 57 percent of college graduates but only 19 percent of computer science graduates (National Center for Education Statistics, 2019). This, in turn, translates into a low share of female participation in technology occupations. Even though women represent about 45 percent of the workforce in Argentina and Colombia, the share of female employees in technology occupations is 22 and 20 percent, respectively (International Labor Organization, 2014).<sup>1</sup> There are several reasons why this underrepresentation of women in technology is concerning and needs attention. From an economy-wide point of view, this implies a potential misallocation of talent across sectors that could impede aggregate growth (Hsieh et al., 2019). From an employer point of view, firms miss a diversified workforce, potentially impeding productivity gains (Sax et al., 2017). Finally, and most relevant for our study, from the worker point of view, this implies that women are missing in occupations that pay higher wages (Blau and Kahn, 2017) and that rank the best in terms of amenities (Goldin, 2014). The most salient amenities in the technology sector are related to job flexibility: these jobs are better suited for flexible working hours, work-from-home, and part-time arrangements. These are particularly relevant for women of childbearing age, who are the focus of this study.

In the present paper we use a randomized control trial to analyze whether digital courses can have the to women with the skills to ac- training potential equip necessary cess the _tech_ sector. We evaluate the causal effects of a high-quality, intensive part-time coding bootcamp on skill acquisition, employment and educational outcomes on a sam-

> 1In the United States, the share of females in the labor force is 47 percent, versus 25 percent in Computer Science related occupations (U.S. Bureau of Labor Statistics, 2019).

1
ple of young female beneficiaries. To do so, we randomly offer applicants to an oversubscribed coding bootcamp in Buenos Aires, Argentina, and in Bogotá, Colombia, a spot in the course together with a scholarship to cover most of its tuition costs.

The bootcamps were carefully designed considering the following dimensions. First, the selection of well-established coding schools with a proven track record and experience in providing high value-added bootcamps was key to create a high return course valued by the market. Second, the scholarship that covered most of the tuition costs aimed to tackle a participation constraint originated by the high costs of these courses. Third, the coding schools in both locations worked closely with gender specialists to define strategies to make bootcamps more gender inclusive, key in order to increase participation in a sample of female beneficiaries. Last, the eligibility criteria were designed to attract a sample of women with high motivation, willingness to complete the course, and high potential to benefit from it.

Even there is a recent but extensive literature on the effects of vo- though relatively cational training programs,<sup>2</sup> most of the interventions evaluated focus on a population of low-income youth or unemployed individuals, with the main objective of the program being to increase employment. The evidence on how effective these programs are at achieving their main objective of increasing employment is mixed. There exist several successful programs where training increases employment and do so for the long-term (Attanasio et al., 2011, 2017; Barrera-Osorio et al., 2020; Kugler et al., 2020; Silliman and Virtanen, 2019), but there are others that either do not achieve an increase in employment or earnings (Hirshleifer et al., 2015; Card et al., 2011; Cho et al., 2013), or which impacts fade away in the middle- or long-term (Acevedo et al., 2017; Alzúa et al., 2016). Our paper will contribute to this literature by analyzing yet a new type of vocational training which, to the best of our knowledge, has not been studied before. The most salient distinctive feature of the program evaluated in this paper is the population under study, which entailed a population of young females with a high level of educational attainment and who are already participating in the labor market at high rates.<sup>3</sup> Its main objective consists of relaxing an important barrier of entry of these women into a _tech_ employment, and it does so by providing the skills sought by employers in the sector.

> 2McKenzie (2017) provides an excellent review of the evidence.

> 3As explained in Section 2 of the paper, the program targeted women broadly, with a minimum eligibility criteria of high school completed. Despite this, women who concluded the whole selection process presented high levels of education and high levels of participation in the labor force.

2
We randomly assign a spot in the course together with a scholarship to cover 65 percent of the cost of a computer coding course in Argentina, 80 percent of the cost in Colombia. Given a limitation on the sample sizes, we analyze the effects of the training on the pooled sample of beneficiaries in both countries.<sup>4</sup> We analyze the effects of the program on two sets of primary outcomes of interest: skills, and employment in technology. We find strong impacts of the program on skills, measured by an exam that resembles a coding test that an employer would apply during an interview to a candidate for an entry-level coding job. We also find positive and significant effects of the program on the probability of getting a job in technology. Importantly, we compare key characteristics of these jobs to the other jobs in our sample (which we call non-technology jobs). We show that even though technology jobs pay the same rate, they are more likely to offer flexible hours and workfrom-home arrangements (in line with Goldin (2014)), and they generate a higher job satisfaction when compared to non-technology jobs. We interpret this as a net improvement in overall job quality.

The analysis in this paper focuses on the short-term impacts of the program, since our follow-up data gathered information referring to February 2020, between 2 and 5 months after program completion. Moreover, we also gather employment information referring to May 2020. This provides us with a unique opportunity to measure whether the program effects make beneficiaries more resilient to the onset of a COVID-19 labor market crisis.<sup>5</sup> We hypothesize that the bootcamps could equip beneficiaries with an alternative source of employment and earnings should they face an adverse employment situation –either hours, pay cuts, or the loss of the job–. Compared to the control group, the treated group could exploit, for instance, the value added of the bootcamp on their CVs to find a better job, or take advantage of the possibility of self-employment in coding-related jobs, where self-employment and freelance jobs are frequent. The evidence suggests so, especially for the sub-sample of the self-employed.

> 4We worked closely with the coding schools in Buenos Aires and Bogotá and carefully designed the intervention so that it was as similar as possible between both countries. This included making the targeting of the program similar, as well as the content and structure. Minor differences across locations were adopted to adjust to the local context and/or constraints. The amount of the subsidy varies across locations to adapt to the local context, local cost of full tuition, and local costs of living.

> 5Employment data of February 2020 gathers information for a period before the COVID-19 crisis hit Argentina and Colombia. Both countries experienced the outbreak in COVID-19 cases long after the Northern Hemisphere. The first measures of lockdowns in Buenos Aires occurred in March 20, 2020, while in Bogotá it was in March 25, 2020. These lockdowns happened still when both cities had very few cases: by the time this first lockdown took place in Argentina (Colombia), there were only 97 (92) infections, and 3 (1) deaths nationwide. Still by May 2020, the COVID-19 outbreak was at its earliest in both countries.

3
Besides analyzing the effect of the training course, we aimed to understand potential ways to increase program participation. In particular, we focused on a potential barrier of entry into the computer and technology sector that could be related to social norms in an extensively male dominated field. Motivated by previous literature that finds that women shy away from fields in which they experience minority status (Buser et al., 2014; Reuben et al., 2017; Bostwick and Weinberg, 2018; Shan, 2020), we nested a complementary intervention within the treated group. We randomized half of the beneficiaries of the scholarship into a paired treatment that consisted of providing them with the contact information of another female classmate in the treatment group. The contact information of this peer was provided together with the scholarship offer, and examples of how they could take advantage of such partner were given to encourage their interaction. The theory that guided this intervention is that by receiving the encouragement to contact another female peer, beneficiaries of the lottery could have perceived that the training course was female-friendlier, and this might have had a positive impact on the decision to enroll. We fail, however, to detect a significant effect of this component of the intervention. We believe that the absence of this effect can be mostly due to two features. First, this treatment arm might have been under powered. Second, the outreach campaign for the program made a strong focus on gender inclusiveness and on the fact that this was a program targeted to women. In that context, an additional encouragement to form a team with another female student might have had only a very small marginal value added to the attraction campaign.

The rest of the paper proceeds as follows. In Section 2 we describe the program and the experimental design. In Section 3, we describe the data and present randomization tests to validate our experimental design. We present the results in Section 4, and conclusions in Section 5.

# **2 Program Description and Experimental Design**

## **2.1 Program Design**

We worked closely with two well-known coding schools -Digital House in Buenos Aires and Bogotá Institute of Technology in Bogotá- to design and implement two ready-towork coding bootcamps. While the budget of the program allowed for only 150 vacancies 4
in each location, our sample calculations required a larger number of individuals in the treatment group. This is why, as specified in the pre-analysis plan, we pooled samples in both locations for our main analysis. This required that both the content as well as the targeting of the bootcamps needed to be designed to be almost the same for both cities, with small variations only to allow for adjustments to the local context.

The bootcamps were part-time high-quality rapid-skills courses that prepared students to qualify for an entry-level coding job shortly after its completion. Courses entailed a mandatory computational training component and an optional soft-skill and career development module. The mandatory computational training component entailed basic programming skills based on labor market demands, including JavaScript, Hypertext Markup Language (HTML), Cascading Style Sheets (CSS), and other platforms. Students were also required to complete various practical projects in a workshop setting. The optional softskills component entailed professional development workshops focusing on confidencebuilding, leadership, communication and presentation skills. Workshops on job market strategies were also offered, covering topics like building a LinkedIn profile, resume development, job interviews, and teamwork. Personal mentoring sessions with a coach were included in this optional component as well.

The program was carefully designed drawing on lessons learned from earlier pilots in Colombia, Lebanon and Kenya (World Bank, 2018). The main lessons pointed towards the importance of a strong attraction and outreach campaign to attract enough candidates interested in participating in the bootcamps,<sup>6</sup> the need for a solid understanding of participation constraints that women face, the quality and expertise of the bootcamps providers, the relevance of the curriculum, and the identification of the target of participants who can benefit from the intervention.

First, in order to lead to a high return, bootcamps needed to provide the right set of skills valued by the market. In this regard, the selection of well-established coding schools with a proven track record and experience in providing high value-added bootcamps was key. Second, tuition costs to attend the bootcamps at these high-quality schools are high. In Argentina (Colombia), the monthly tuition costs were equivalent to about 1.2 (0.7) times the mean monthly labor earnings in our sample. The scholarship to cover most of the

> 6The pilots implemented in Lebanon and Kenya counted with only 15 and 18 students respectively, making it unfeasible to implement any quantitative evaluation. The pilot in Colombia included a total number of 281 enrollees, of which 120 were selected as beneficiaries, and 161 as non-beneficiaries. The impact evaluation for this pilot did not find significant results of the program on its main outcomes of interest. The study cannot rule out that this could be due to a sample with an insufficient size (World Bank, 2018).

5
tuition costs aimed to tackle this dimension.<sup>7</sup> Third, bootcamps are male dominated environments. Historic enrollment data provided by the coding schools indicates that, on average, 80 percent of students are male.<sup>8</sup> In order to address this issue, the coding schools in both locations worked closely with to define to make more inclusive. In or- gender specialists strategies bootcamps gender der to increase female participation, this aspect was explicitly highlighted and publicized in the attraction campaigns (explained in detail in section 2.2). Fourth, even conditional on participation, previous pilots of bootcamps identified a prevalent problem of retention for female participants (Hammond et al., 2018). For this, the eligibility criteria (explained in section 2.2 as well) was designed to attract a sample of women with high motivation, willingness to complete the course, and high potential to benefit from it.

Last, logistical issues aimed at increasing program participation were also considered in the design. Participants could choose location, shift, and between two learning modes. Each city counted with two locations of the learning center and three different class shifts (morning, afternoon and evening). One learning mode included mostly face to face classes that lasted at most 3 and a half hours a three times a week. The other one was a day, "blended mode", a mix of face to face and online classes.<sup>9</sup> The part-time design of the bootcamp was key in order to allow participants to continue working or studying another career, if needed.

In Buenos Aires, the bootcamp lasted 20 weeks (with a total of 210 hours of instruction). Most classes started in early April and finished in late July 2019. The last class started in mid-May 2019 and finished in mid-September 2019. In Bogotá, the bootcamp lasted 16 weeks (with a total of 170 hours of instruction). Most classes started in early August and finished in late November 2019. The last class started in mid-September and finished midDecember 2019.<sup>10</sup>

> 7Maffioli et al. (2020) show that even though most entrepreneurs have a positive willingness to pay for training, demand falls sharply as price increases: 76 percent of the entrepreneurs attend training when it is free, but only 43 percent attend when they are charged one-quarter of the cost, and only 11 percent when charged the full cost. The paper also shows that charging a higher price increases attendance among those who pay, suggesting a psychological effect where paying for training makes entrepreneurs value it more.

> 8The previous pilots counted with a share of female students of 30 percent in Lebanon, 25 percent in Kenya, and 28 percent in Colombia (World Bank, 2018).

> 9Considering these different locations of the learning centers, shifts and modes, there were a total of 12 classes available for enrollment in Buenos Aires and 10 in Bogotá.

> 10For a detailed timeline of implementation and research activities in each country, please see Figure A.3 in the Appendix.

6
# **2.2 Outreach and Eligibility**

With the objective of attracting a large number of applicants, considerable effort was put into defining and monitoring strong advertising campaigns and clear application processes.

The first step in this attraction campaign consisted of publicizing the bootcamp through social networks (Facebook, Instagram, Google, YouTube Ads, and LinkedIn) and local news outlets. The campaign included concise banners and images accompanied by a link to a landing page with more detailed information about the program and enrollment process. Information included the fact that there were only 150 vacancies (with their respective scholarship) available in each city and that, expecting over-subscription, the implementer was going to assign them randomly. In that same landing page, those interested in participating in the program were able to begin the application process.

The application process and eligibility criteria included the following: (i) minimum of high school diploma; (ii) completion of a questionnaire; (iii) pre-work assignment completion; and (iv) a final exam. The questionnaire included information on demographic characteristics, education, and labor situation of the candidates, among other relevant information. This was used to gather baseline data. The pre-work consisted of a self-learning module to ensure that all participants had the minimum level of computer knowledge required to benefit from a coding bootcamp. This was not coding-specific, since no previous coding was required in this program. Participants had about 3 weeks to complete this. The final exam was taken on-site in Colombia (written exam) and Argentina (oral exam with interview format). This final exam had the intention of evaluating the minimum knowledge acquired through the pre-work. Participants who finished this application process satisfactorily were deemed eligible to participate in the lottery to receive the scholarship.

Figure 1 shows the number of participants that initiated the application process and those who finished it. A total of 803 (393 in Argentina and 410 in Colombia) applicants constitute our sample of eligible candidates.

# **2.3 Experimental Design**

The experimental design was identical in both countries, where we randomly assigned applicants who enrolled in the coding bootcamps into different treatment arms. First, separately by city, we randomly select half of the eligible population to receive a spot in the 7
course, together with a scholarship that covered 65% of the tuition costs in Buenos Aires, 80% in Bogotá.<sup>11</sup> Second, nested within the treated group, half of the applicants randomly received, in addition to the scholarship, the encouragement to contact another female peer classmate before enrolling. The contact information of this peer was provided together with the scholarship offer, and examples of how they could take advantage of such partner were given. Participants in this paired treatment were encouraged to contact their peer before enrollment. During the course of the bootcamp, they received two reminders about the information of their peer and about how to take advantage of this interaction.<sup>12</sup> The lottery was implemented in two stages to ensure enough enrollment. Enrollment consisted of accepting the scholarship and paying the remaining 35% (20%) of the tuition cost in Argentina (Colombia). The program committed to support 150 scholarships in each and it was for the statistical of the to achieve this country, necessary power study target. Since it was anticipated that some applicants would not enroll even after being selected, the solution was to run a lottery in two rounds. In each country, a first round of the lottery identified 150 participants as beneficiaries of the lottery, while the remaining eligible participants were waitlisted. The 150 beneficiary applicants were given two weeks to secure the payment of the reminding tuition cost. Enrollment for this first round was 74% (71%) of the 150 who received the offer in Argentina (Colombia). A second round of the lottery communicated the final outcome -either won the lottery or not- to those in the waiting list. Beneficiaries in this second round of the lottery were also given two weeks to secure the payment of the reminding tuition cost. Appendix A.2 explains in detail how this lottery was executed. We adjust our estimates to account for this method of random assignment following De Chaisemartin and Behaghel (2020).

After completing the two rounds of the lottery in both countries, out of the eligible sample of 803 participants, a total of 402 received the offer of the scholarship. Within this 192 received the additional to contact another female class- group, encouragement peer mate, while 210 received the scholarship only. The control group is composed by 401 applicants who did not receive neither the scholarship nor the encouragement to contact a peer. Figure 2 summarizes the treatment arms in the experimental design. 11For Argentina, the remaining 35% was AR$30,000, while for Colombia the remaining 20% was CO$600,000. This amounts for about 2 times the mean monthly labor earnings for our sample in Argentina, about half the mean monthly labor earnings for Colombia.

> 12Appendix figures A.4 and A.5 provide the letters received by applicants in the scholarship-only and in the paired treatment groups.

8
# **3 Data**

## **3.1 Data Collection**

A baseline questionnaire was collected before performing the random allocation of the scholarship offer. This questionnaire included socio-demographic characteristics of the applicants, educational attainment, questions related to coding skills and previous coding knowledge, employment variables (status, industry where the applicant works, wages), and social skills as measured by the Big Five Inventory by Soto and John (2017). We gathered the tax identification numbers of respondents, which may be used in the future to analyze long term outcomes using administrative data. Given the scale of our experiment and power calculations, we applied the baseline questionnaire to all 803 eligible candidates.

Originally, a first follow-up face-to-face data collection was planned to take place after about one year of the program completion, with the objective of measuring short-term impacts on skill acquisition and employment dynamics. This would have been during September 2020 in Buenos Aires, December 2020 in Bogotá. However, these activities needed to be re-adjusted given the outbreak of the COVID-19 pandemic. In order to still capture the short-term impacts of the program, we designed and implemented an online survey during May and June 2020. The objective of this online survey was threefold: first, it was important to gather information about labor market outcomes of program participants right before the COVID-19 global crisis affected labor markets in both cities. For this, the period of reference of all employment questions in the survey dated back to February 2020. Second, information on educational outcomes and skills was collected in order to measure short-term impacts of the program on these indicators. Third, a specific module related to COVID-19 was administered in order to assess whether bootcamps participants were somehow in a better position than non-participants to tackle the crisis. This module included a question on resilience to a crisis in the job market generated by the COVID-19 outbreak, as well as employment information relative to May 2020, when the questionnaire was administered. Given the challenging times during which this online questionnaire was collected and in order to maximize response rates, we designed an instrument that aimed to last no more than 20 minutes to complete.<sup>13</sup> Data Appendix A.1 provides

> 13The average response time ended up being 22 minutes.

9
further details on how we collected baseline and follow-up data.

We were able to re-interview 724 of the 803 individuals in the sample, achieving a 90.2% response rate. An attrition rate of 9.8% compares very favorably to the attrition rates found in other vocational training studies<sup>14</sup> or even in labor market administrative surveys for developed countries.<sup>15</sup> Response rate for the treatment group was 93.3%, while for the control was 87.0%. small in Table 1 shows that this group Although magnitude, difference across treatment groups is statistically significant. We deal with this differential attrition in several ways. First, in Table 3 we show that the sample of non-attriters is extremely similar in pre-treatment observable characteristics to the whole sample. Moreover, we show that pre-treatment covariates remain balanced between treatment groups for the sample of non-attriters: all the differences between treatment and control groups show roughly the same magnitudes and statistical significance when compared to the whole sample. Second, we investigate whether any pre-treatment observable characteristic correlates with the probability that an individual attrits the sample. We estimate a linear probability model in which we regress a dummy that takes the value of 1 if individual attrits, 0 if she remains in sample on treatment status, randomization strata fixed effects, and baseline characteristics.<sup>16</sup> Figure 3 shows the point estimates and confidence intervals for each variable in such regression. In this figure, we can observe that the only variable that significantly predicts attrition is treatment assignment, with no systematic correlation with any other pre-treatment characteristic.

Given that the differential response rate is small in magnitude, that it is not correlated with pre-treatment characteristics, and that the non-attriters sample shows the same balance as the whole sample, our main estimates do not adjust for attrition. Reassuringly, however, in section 4.7 we show that these estimates are robust to adjusting for differential attrition. We do so by implementing bounds by Behaghel et al. (2015).<sup>17</sup>

> 14See Table 1 (pages 132 and 133) in McKenzie (2017) for the attrition rates in 15 other vocational training studies in low and middle income countries.

> 15The attrition rate for the U.S. Current Population Survey, for example, is about 20%.

> 16Baseline characteristics include age, marital status, educational attainment, a dummy that indicates whether the participant has taken a previous coding course, a dummy that indicates whether the participant specialized in a STEM career during her education, employment status, whether the individual was employed in the formal sector, weekly hours employed, and labor earnings.

> 17Since we foresaw this differential attrition, we developed a thorough field protocol that collected information on how many call and email attempts were made for each subject in the sample.

10
# **3.2 Sample Descriptive Statistics**

We begin by reporting basic pre-treatment descriptive statistics regarding demographics, education, skills and labor market outcomes for our sample in Table 2.<sup>18</sup> We do not distinguish between treatment and control groups in this section, but we show statistics for our pooled sample and separately for the sample in Argentina and Colombia. We do this to get a sense of how our sample compares across countries and, within country, to the overall target populations in each location. Following a similar criterion of the program targeting of beneficiaries, for the overall population in Argentina, we compute descriptive statistics for women between 18 and 40 years old, living in Buenos Aires city and Greater Buenos Aires Area. We get this data from the 2018 Permanent Household Survey ( _Encuesta Permanente de Hogares_ , INDEC). Equivalently, for Colombia we compute descriptive statistics for women between 18 and 50 years old in Bogotá. We get data from the 2018 National Household Survey ( _Gran Encuesta Integrada de Hogares_ , DANE).

2. Several facts are worth describing from Table Regarding demographics of our sample, the average age of eligible women in our sample is about 29 years old. Even though only 19 percent of our sample are mothers, the mean age in our sample is remarkably similar to the mean age of mothers at first childbirth, which is 29.5 years old for Buenos Aires (Direccion General de Estadistica y Censos, 2019), and 27 years old for Bogota (OECD, 2019). About 14 percent of women in our sample are married. The household income distribution in our sample is somehow skewed towards upper deciles of the income distribution, with 44 percent of our individuals having a household income in deciles 4-7 and 38 percent in the upper 3 deciles. Age and marital status of eligible candidates are extremely similar across bootcamps locations. Our sample in Argentina has a somewhat smaller share of mothers relative to Colombia, and a household income distribution relatively skewed to the right.

To summarize education, we show the percentage of individuals who have a college degree in our sample. This is 61 percent, and is very similar between Argentina and Colombia. Last, employment variables show that 83 percent of our sample is employed at baseline, with 22 of them being self-employed, and about 60 percent of them being employed in the formal sector.<sup>19</sup> On average, individuals in our sample work about 27 hours a week, and their wage and salary earnings are US$ 392 a month. While the employment figure

> 18Data Appendix A.1 details how we construct each variable in our data.

> 19Our self-employed and formal sector dummies assign zeros for those not employed.

11
and weekly hours worked are similar between Argentina and Colombia, the sample in Argentina shows a lower share of self-employed and a higher share of formal employment compared to Colombia. Wage earnings (in December 2018 dollars) are lower for Argentina.

Last, when we compare our samples to their respective overall target population in each country, we observe that our samples are somewhat positively selected into the program. In both countries we observe that the share of mothers is significantly lower relative to the population, with a share of college degree educated eligible candidates being significantly higher. Employment variables also show a positive selection in terms of employment status, formality and earnings. We believe this positive selection was the result of the selection process explained in section 2.2. It is important to note here that even though the eligibility criteria were set broadly (high-school completed), the selection process (mostly the completion of the pre-work assignment) identified candidates with a high willingness and motivation to participate in the bootcamp, which in turn resulted in a positively selected sample as described here. We will come back to this when we interpret our results.

# **3.3 Randomization Checks**

The randomization procedure made use of the information in this baseline. To ensure a balanced sample, randomization was stratified (Duflo et al. (2007); Bruhn and McKenzie (2009)). The stratification was done based on educational attainment (sample divided between those with less than college versus college or more), motherhood status (being a mother or not), and previous employment in a job related to coding (sample divided between those who had such experience and those who had not). Repeating this by country, this gives a total of 2 _×_ 2 _×_ 2 _×_ 2 = 16 randomization blocks.

We present pre-treatment differences in demographic characteristics, education, skills and labor market outcomes between the treatment and control groups in Table 3. The table reports pre-treatment means of each variable for the treatment and the control groups in columns 1 and 2 respectively. Column 3 reports the difference in each variable between the treatment and control groups and its estimated robust standard errors. We control for randomization strata fixed effects when we estimate these differences.

Overall, we notice that the two samples are remarkably balanced, indicating that the randomization worked well. The only exception is a small imbalance in the wage earnings 12
between the treatment and control group, where the treatment group shows earnings that are approximately 10 percent lower than the earnings of the control group. Even though the difference is marginally statistically significant (p-value of 0.076), all our estimations include this variable as covariate. Moreover, when we conduct a test of joint significance of differences of all pre-treatment characteristics, we do not reject the null hypothesis that all characteristics between treatment and control group are the same (F-test=0.68 with a p-value of 0.835).

Given that Table 3 uses only data from our surveys, the set of variables we show there is richer than the one described in Table 2, particularly for variables regarding education and skills. A few additional things are worth mentioning about our sample characteristics that were not described in section 3.2. We observe that about 30 percent of our sample has specialized on a STEM degree<sup>20</sup> during their education. The proportion of the sample that holds a degree in Computer Science is significantly lower: only 5.5 percent. About a third of our sample has taken a coding training course before the bootcamp. Importantly, all these variables that measure skills as well as the Big 5 social skill index are perfectly balanced between treatment and control groups in our sample.

As mentioned in section 3.1, we repeat the balance tests for the sample that responds the follow-up online survey. Results are shown in columns 4-6 in Table 2. Three features are worth mentioning. First, characteristics of the overall sample remain the same between the whole sample and the sample of non-attriters. Second, as shown in Table 3, the sample of non-attriters continues to show balance between treatment and control groups.<sup>21</sup> Last, the F-test of overall significance keeps a similar magnitude and statistical significance. Given this, our main estimates do not adjust for this attrition. Moreover, in Table 10 we show that our main estimates hold even after adjusting for differential attrition by implementing the bounds by Behaghel et al. (2015).

> 20STEM degrees are Biology, Biochemistry, Computer Science, Math, Physics, and any specialization of Engineering.

> 21Indeed, the only imbalance present in the whole sample disappears in the sample of follow-up nonattriters.

13
# **4 Program Effects**

## **4.1 ITT and LATE Estimations**

In the main specifications of this paper, we report Intention to Treat (ITT) and Local Average Treatment Effect (LATE) estimates of the bootcamp on different outcomes of interest. ITT estimates are obtained from the following regression: where _Ti_ is an indicator that takes the value of 1 if the individual _i_ is randomly allocated a spot in the bootcamp with a scholarship offer, 0 if not. Unless otherwise specified, regressions that control for pre-treatment characteristics include the following in vector _Xi_ : age, marital status, educational attainment, a dummy that indicates whether participant has taken a previous coding course, a dummy that indicates whether participant specialized on a STEM career during her education, employment status, whether the individual was employed in the formal sector, weekly hours worked, and wages and self-employment labor earnings. A vector of 16 randomization strata fixed effects is included in _Si_ , and _εi_ is an error term.

_yi_ is the outcome of interest. We focus on two sets of primary outcomes. First, the primary outcomes referring to skills are the score in the coding exam, and an indicator that takes the value of 1 if individual passes the exam (i.e., scores higher than 4) and 0 otherwise. The second set of primary outcomes of interest refers to access to the technology sector. We analyze the impacts of the bootcamp on an indicator that takes the value of 1 if, at follow-up, the individual is "in the technology sector", 0 if not. We measure being "in the technology sector" as being either working or employed at a job where the individual performs coding tasks<sup>22</sup> or studying a computer science (or related) degree.<sup>23</sup> We then analyze these two outcomes separately. We analyze the impacts of the program on getting a coding job, and on studying a computer science degree. Unless otherwise noted,

> 22As explained in detail in Data Appendix A.1, when we asked this information in the interview, we were explicit about what a coding task meant. We indicated that by "coding" we meant programming in some computer language such as Ruby, Python, JavaScript, HTML, CCS, iOS, C ++, Laravel, or similar. We explicitly indicated that the use of a computer for any other software such as Microsoft Excel, for example, was not considered coding.

> 23Computer Science degree includes Computer Science and different variations of Computer Engineering.

14
throughout the paper we use a Linear Probability Model whenever we analyze binary outcomes.<sup>24</sup> The parameter of interest in equation (1) is _β_<sup>_ITT_</sup> . Under random assignment, this parameter represents the causal effect of the program, i.e., of the spot offer.

In order to identify the effect of the bootcamp, we estimate the LATE using equations (2) and (3) below.

In equation (2), _Bi_ is an indicator variable that takes the value of 1 if the individual gets the offer, enrolls and graduates from the course, 0 otherwise. _B_<sup>ˆ</sup> _i_ in (3) is the fitted value of _Bi_ , and _β_<sup>_LATE_</sup> estimates the local average treatment effect of the bootcamp.

Before proceeding with the analysis of the effects of the program, in the next section we discuss program compliance.

# **4.2 Compliance**

Out of the 402 spot offers, 283 individuals (70.4%) enrolled in the course. Enrollment in the bootcamp implied paying the remaining 35% (20%) of the tuition cost in Argentina (Colombia). Only 2 individuals from the control group enrolled in the course. Out of the 285 enrolled, 85% (241 students) finished the course and graduated.<sup>25</sup> According to administrative data, most dropouts occurred during the first month of bootcamp.<sup>26</sup> A few additional things are worth mentioning about compliance in our context. First, for the LATE estimates we define compliance (our _Bi_ indicator in equation (2)) as enrolling and graduating from the course. Second, even though only 2 individuals from the control group enrolled in our course, about a third of the control group enrolled in another coding bootcamp after our random treatment assignment. We keep our definition of program compliance limited to the course offered in this particular intervention. According to our

> 24Section 4.7 presents the robustness of our main ITT estimates when we use a conditional logit model for binary outcomes.

> 25This graduation rate compares very favorably relative to previous pilots and similar bootcamps (Hammond et al., 2018).

> 26Of the 44 dropouts, 40 alleged personal reasons or incompatibility with work schedules. The remaining 4 did not provide a reason.

15
follow-up data, most of these other coding bootcamps are free, limited and shorter alternatives compared to the course we evaluate here. Moreover, from a revealed preference point of view, eligible candidates preferred our course to begin with, and enrolled in these alternatives only after not being randomly selected to get the scholarship.

In order to analyze what drives program enrollment and completion, we estimate the following equation within the sample of treated individuals: where all variables have been defined previously except for _Pairedi_ , which is an indicator that takes the value of 1 if individual _i_ in the treatment group is randomly assigned a pair. In particular, when estimating equation (4) we are interested in the estimate of _γ_ to study whether the paired treatment encouragement design has any effect on enrollment or completion.

4 4. Estimates from an OLS specification of equation ( ) are presented in Table Despite showing positive point estimates, ˆ _γ_ in that table is imprecisely estimated, meaning that we cannot conclude that the paired treatment encouragement has a positive effect on program take-up or on graduation. The absence of this effect can be due to different factors. First, it could be that this treatment arm was under powered.<sup>27</sup> Second, regarding the absence of an impact on enrollment, as explained in section 2, we have to bear in mind that the outreach campaign for the program made a strong focus on gender inclusiveness and on the fact that this was a program targeted to women only. In that context, an additional encouragement to form a team with another female student might have had only a very small marginal value added to the attraction campaign. Last, regarding the absence of an impact on graduation, the courses ended up being formed by about 50 percent female students, versus the historical 20 percent. In this context, an organic study group with another female classmate must have been easier to form, even for the sample who received the individual treatment.

Given this, in what remains of this paper, ITT and LATE estimates focus on the impacts of the spot offer only.

> 27Power calculations for these outcomes assumed an MDE of 0.2 standard deviations.

16
# **4.3 Impacts on Skills**

We first investigate whether the bootcamp actually provided participants with the skills needed to get a job in technology. For this, we embedded a coding exam in the follow-up survey. This test was designed together with the learning and professional development team in both learning centers. The questions resembled questions that an applicant would get in a job interview for an entry-level coding job. Indeed, the assessment was composed of a pool of questions that alumni from the bootcamp in each location received in real job interviews. The exam included 8 multiple choice questions, and respondents were given 4 minutes to answer those before the questionnaire advanced automatically to the next section.<sup>28</sup> Out of these 8 questions, 3 were "minimum coding knowledge" type of questions. The remaining 5 were harder questions where a screenshot of a coding script was shown and the respondent needed to spot the error. Questions were shown in random order to respondents. The maximum score in this exam was 10 points. Each of the three basic questions were worth 1 point each, 3 of the five were worth 2 points each, and the remaining 2 hardest questions were worth 0.5 points each. A score of 4 was considered a pass. From an interviewer point of view, a score of 4 in a four-minute exam means that the candidate showed -even under time pressure- the minimum knowledge by answering at least the three basic questions and one additional hard question correctly. This passing score of 4 considered a failed test one that only answered the three basic questions correctly. Answering two of the hard questions correctly was considered a pass even if the respondent did not answer any other question. Appendix section A.3 shows examples of the questions asked.

To get a first look at the exam scores, Figure 4 shows the score distributions by treatment group. In this figure, we can observe a shift to the right in the exam score distribution of the treated group, compared to the control group. In particular, relative to the control, we observe a much higher share of individuals scoring 5 or more points in the treatment group.

Panel A of Table 5 presents the estimates of the impacts of the program on skills. Columns (1) and (2) in that table show the ITT estimates of the program effects. Columns (3) and (4) show LATE estimates. The last column of the table shows the mean of the control group for comparison. Odd numbered columns only control for randomization strata

> 28We applied this criterion in order to minimize survey fatigue.

17
fixed effects, while even numbered ones show results from specifications that control for pre-treatment covariates.

We investigate impacts on two outcomes: the score in the exam, and the probability of passing it. Results show a strong positive effect of the program on skills. LATE estimates show that the bootcamp increases the exam score by 1.5 points, which represents a 45 percent (0.6 standard deviations) increase relative to the control group. Importantly, the bootcamp increases the probability of passing the exam by 20 percentage points, which represents a 50 percent (0.4 standard deviations) increase relative to the control.

These strong positive results are important in our context for several reasons. First, and as explained in section 4.2, about a third of the control group took a coding course other than the one offered by this intervention. Given that we define compliance as participating in our bootcamp only, this means our LATE estimates still compare our compliers in the treated group against a control that still may have acquired _some_ level of skills through other alternative courses. However, and according to follow-up data, these other courses were shorter, free and mostly online alternatives of the one evaluated in this study, and the positive effects we observe on skills corroborate that the quality and careful design of the intervention matter for observing the intended effects. We provide some evidence of this superior quality of the bootcamp in the next subsection 4.3.1.

Second, even though our research design does not allow us to study whether these courses work through a credentials signaling mechanism, a positive effect on skills is important to understand that even though this could be at play, the training provides important skills that go beyond whatever signaling effect that the course could have on the beneficiaries’ CVs. This is particularly relevant for our next section, in which we show that the program is also effective at making individuals transition to a job in technology.

# **4.3.1 Evidence of bootcamp quality**

As described in sections 4.2 and 4.3, about a third of the control group (who did not get the offer to participate in the bootcamp) ended up enrolling in alternative courses. Despite that, according to follow-up data, these other courses were shorter, free and mostly online alternatives. Information coming from the performance of individuals in the coding exam provides a good opportunity to compare the quality of the bootcamp evaluated in this study with these other alternatives.

To explore this, we compare the exam scores of the compliers in the treatment group 18
(those who enrolled and finished the bootcamp) with individuals in the control group who took an alternative course. Figure 5 shows the distributions of such scores. When we compare the scores, the treated group shows a score that is about one point higher (18 percent, 0.3 standard deviations) relative to the control group ( _p_ = 0 _._ 011). This can be taken as a piece of evidence that may indicate that the bootcamp evaluated in this study is of higher quality than the alternatives taken by some individuals in the control group.

# **4.4 Employment in Technology**

Having shown that the program has strong positive effects on skills, the next step is to investigate whether beneficiaries actually use the program as a mean to help them transition into the technology sector. In order to understand this, we analyze the program effects on three outcomes of interest. First, a dummy variable that indicates whether, in February 2020 (5 months after the intervention ended in Argentina, 2 months in Colombia) the individual is working on a job that requires coding (we call this variable "In _tech_ working"). Second, a dummy that indicates whether at follow-up the individual is acquiring formal tertiary or university education on a computer science related field (we call this variable "In _tech_ studying"). Last, we create a dummy that subsumes the previous two by taking a value of 1 if the individual is either working or studying in tech, 0 otherwise (we call this variable "In _tech_ (working or studying)").<sup>29</sup> Data Appendix A.1 explains in detail how these variables were measured and constructed. As described in section 3.1, the employment outcomes that we analyze in this section and in section 4.5 use retrospective data collected in May 2020 but referring to February 2020. This is to identify the bootcamp impacts on the main employment outcomes before the COVID-19 outbreak affected labor markets in Argentina and Colombia.<sup>30</sup> Panel B of Table 5 presents the estimates of the impacts of the training on these outcomes. We observe that the program increases the probability of working at a technology job by 9.2 percentage points, which represents a 38 percent (0.2 standard deviations) increase relative to the control group. While we do not observe an impact of the program on the acquisition of formal tertiary or university computer science education, we do find

> 29Being working or studying in _tech_ are not mutually exclusive, and this why, as observed in Table 5, the means of the first two dummies add up to a fraction slightly larger than the mean for the third one. We did not pre-specify this third outcome.

> 30Both countries experienced the outbreak of COVID-19 cases long after the Northern Hemisphere. The first measures of lockdowns in Buenos Aires occurred on March 20, 2020, while in Bogotá it was on March 25, 2020. These lockdowns happened still when both cities had very few cases.

19
that the bootcamp increases the probability of being in the _tech_ sector -either working or studying- by 12.7 percentage points, which represents a 48 percent (0.3 standard deviations) increase relative to the control group.

Since our results indicate that our main impacts come from the employment in technology outcome, we elaborate on these results further. A natural question to ask is: are technology jobs better compared to other jobs in our sample? If so, in which way? In order to understand this, we compare key characteristics of technology jobs to the other jobs in our sample (we call them non-technology jobs). To do this, we perform this descriptive analysis at the job-level, meaning that each observation in this analysis is a job instead of an individual.<sup>31</sup> Table 6 presents the results of this comparison analysis. The general takeaway of this analysis indicates that even though technology jobs pay a similar rate, they show better work arrangements and higher job satisfaction when compared to non-technology jobs. In satisfaction in is 12 to non- particular, job technology jobs percent higher compared technology jobs. This goes in line with the fact that they offer the possibility of working from home and flexible hours at a higher rate: technology jobs are 24 percentage points more likely (or 43 percent) to offer work-from home arrangements and flexible hours. Last, we observe that the pay rate (in hourly amounts and total compensation) is similar across job types, with yearly earnings and compensations being slightly higher in technology jobs, with a difference that is not statistically significant.

These findings align with the ones highlighted by Goldin (2014) and are particularly relevant to understand the importance of the intervention. In particular, given that the study includes a sample of young women who are right at the childbearing age, the feature of the possibility of working from home and flexible hours is of relevance when studying whether they indeed are transitioning to a job that better suits their needs.

# **4.5 Mechanisms and Secondary Outcomes**

In this section, we explore mechanisms that may be behind our positive results on employment in technology, as well as secondary outcomes.

A first natural mechanism by which program participants may be getting jobs in coding was already discussed in section 4.3 and relates to an increase of a particular set of skills

> 3183 percent of our sample of employed individuals works in one single job, 20 percent in two jobs, 7 percent in three jobs.

20
that make them more employable. As discussed previously, program participants score significantly higher in a test that resembles an interview for a coding job, and show a significantly higher probability of passing that test.

Beyond this, we explore three additional mechanisms: type of job (as of part- or fulltime), job search behavior, and gender perceptions of the _tech_ sector.<sup>32</sup> Panel A of Table 7 shows the results of our main estimations on this set of outcomes. The first thing to observe from these results is that the positive result observed on the probability of getting a _tech_ job is mostly driven by part-time jobs, with a weekly increase of about 3 hours worked in this type of employment.

Second, we observe no impact of the program neither on job search variables (measured as number of job applications and number of job offers received between baseline and follow-up) nor on individuals’ gender perceptions of the _tech_ sector. The lack of results in this set of outcomes may be indicative of the increase in skills being the main mechanism at play through which beneficiaries of the program increase their employment in a _tech_ job.

Last, we investigate program effects on secondary outcomes: (any) employment and labor earnings. Regarding (any) employment, measured as whether the individual was employed in February 2020, we consider this as a secondary outcome because of the nature of our intervention. As previously explained in section 3, our sample is a sample of individuals who are already participating in the labor market at high rates (with an employment rate of 85 percent), and whose main motivation to participate in the coding bootcamp is to get a particular type of employment, as observed in the positive effect of the program on the probability of getting a job in technology. Second, labor earnings are considered a secondary outcome in this evaluation given its short-term nature. Panel B of Table 7 shows the results of our main estimations on this set of secondary outcomes. Surprisingly, the program has a negative effect on employment: it decreases the probability of being employed by 9.8 percentage points (or 12 percent relative to the control group). This we negative effect seems to be one of labor force participation rather than unemployment: do not observe an impact of the program on unemployment.

In order to further investigate this negative impact of the program on employment, we perform three analyses. First, we estimate the impact of the program on the probability of being in school, measured by an indicator variable that takes the value of one if the

> 32Appendix A.1 details how this variable is constructed.

21
individual declares, at the time of the interview, to be at school pursuing any tertiary or university degree. We do not observe an impact of the program on this outcome.

Second, to shed some light on what the non-employed are doing, we focus the analysis on educational enrollment within this sub-sample. Table 8 presents this descriptive analysis for two subs-samples: those who are non-employed at follow-up, and those who transition from employment at baseline to non-employment at follow-up. Even though the differences between treatment and control group are non-significant (we are using a very small sub-sample), we do observe that, among those non-employed, the treated group is more likely to be enrolled in a tertiary or university degree, with this difference being considerable higher if we focus on computer science degrees.

Third, we perform a treatment effect heterogeneity analysis considering the level of we satisfaction the individual declared to have with her job at baseline. In this analysis, interact the treatment with an indicator variable that takes the value of 1 if the individual declared to be unsatisfied with her job at baseline. Table 9 reports the findings of this analysis. We observe that, while the treatment dummy becomes insignificant, the interaction term is negative and significant ( _p_ = 0 _._ 066). This means that the negative effect on employment is being mostly driven by those who were not satisfied with their jobs at baseline. In other terms, it could be evidence indicating a withholding in labor force participation among those unsatisfied with their jobs before participating in the bootcamp. Since these are very short-term results, it will be important to see what happens to these individuals in the longer-term follow-up.

Regarding the analysis of the effect of the program on labor earnings, we investigate two outcomes. First, we simply estimate the effects on monthly labor earnings. We do not see an impact of the bootcamp on this indicator. Second, we construct a yearly total compensation variable. This is a measure that includes, in addition to labor earnings, a monetized value of the health care and retirement plans, paid sick leave and paid time off, retirement contribution, and yearly bonus, in case the job offers any of these. We do not observe an impact of the bootcamp on total compensation either. This absence of impacts on earnings is in line with findings from the literature that evaluates vocational training programs (McKenzie, 2017).

22
# **4.6 COVID-19**

The results described in the previous sections refer to February 2020, which is before the COVID-19 crisis hit Argentina and Colombia. Both countries experienced the outbreak in COVID-19 cases long after the Northern Hemisphere. The first measures of lockdowns in Buenos Aires occurred on March 20, 2020, while in Bogotá it was on March 25, 2020. These lockdowns happened still when both cities had very few cases: by the time this first lockdown took place in Argentina (Colombia), there were only 97 (92) infections, and 3 (1) deaths nationwide.<sup>33</sup> Since our data collection activities took place during May and June 2020, this gave us a unique opportunity to gather employment information of the interviewees at the moment of the interview, in addition to the retrospective data dating back to February. The objective of this was to compare the employment situation of our sample before and at the onset of the COVID-19 outbreak.

The first outcome of interest related to the COVID-19 development refers to a question that aimed to measure how resourceful and well equipped the individuals felt when having to navigate the job market during or after COVID-19 times. Precisely, interviewees were asked the following question: "On a scale from 0 to 100: How well equipped do you feel you are to navigate the current and future labor market in the midst of a COVID-19 crisis?" Panel C in Table 7 presents the ITT and LATE estimates of equations 1 and 3 on this outcome. We observe that the bootcamp increases this score by about 6 points, which represents a 10 percent increase (.25 standard deviations) relative to the control group.

Then, for those who were employed in February 2020, we compare their situation between February and May 2020. We hypothesize that the bootcamps could equip beneficiaries with an alternative source of employment and earnings should they face an adverse employment situation –either hours, pay cuts, or the loss of the job–. Compared to the control group, the treated group could exploit, for instance, the value added of the bootcamp on their CVs to find a better job, or take advantage of the possibility of self-employment in coding-related jobs, where self-employment and freelance jobs are frequent. We explore this in the next paragraphs.

Figures 6 and 7 present descriptive evidence of the May versus February comparison. Figure 6 presents the case for wage earners, where we show the percentage (within the respective treatment group) of those who got an hour or pay reduction, those who were

> 33More information on the context of the COVID-19 situation during March 2020 in Argentina can be found here (in Spanish). For Colombia, please see this (in Spanish too).

23
laid off, and those who did not see their employment situation change. We do not observe any relevant differences between treatment and control groups. We do observe, however, that independently of the treatment group, about 9 percent of women got laid off by May, and that about 8 percent got their hours or pay cut.

When we do the same analysis considering those who were self-employed ( _freelancers_ ) in February, we do observe a more marked difference between treatment and control group, with the treatment group performing better. Figure 7 shows the proportion (again, within the respective treatment group) of those who saw their employment situation improved (by working more hours and getting higher earnings), those who saw it worsened (by either working less hours and/or getting lower earnings), and those who did not experience any change. We do observe that the treatment group shows better resilience and employment status overall: they are 9 percentage points less likely to have seen their job situation worsen. Breaking this percentage up, the treatment group is 5 percentage points more likely to have seen their employment situation improve, and 4 percentage points more likely to experience the same situation. Even though these differences are not statistically significant (out of the employed at follow-up, freelancers comprise only 23 percent, so these differences are on a sample size of 129 individuals), they can be indicative of the treatment providing more or better resources for those self-employed.

A caveat of this comparison analysis is that, still by May 2020, the COVID-19 outbreak was at its earliest in both countries. In the next follow-up, we will observe the employment situation of the sample after allowing for a longer period of time. We will then be able to better explore whether the bootcamp helped participants cope and deal with any negative circumstances related to a widespread downturn in the labor market.

# **4.7 Robustness and Additional Results**

In this section we present the robustness of our main results to attrition, to adjustment to multiple hypothesis testing, and we present results using a conditional logit model for binary outcomes. Moreover, we compare our main results that use the pooled sample to those obtained by splitting the sample by country.

Table 10 presents the robustness to attrition. We present our pre-specified bounds obtained by the method by Behaghel et al. (2015). We also compare these with the commonly used Lee (2009) method. All our results are robust to attrition, as evidenced by both the upper and lower bounds of our estimates remaining statistically significant. Moreover, we 24
do observe that the method by Behaghel et al. (2015) yields narrower bounds compared to Lee (2009). This highlights the importance of collecting contact information during fieldwork.

Table 11 presents the robustness of our primary outcomes to multiple hypothesis testing. We show the unadjusted p-values used in our baseline ITT and LATE estimations together with the Westfall-Young stepdown adjusted p-values for all our primary outcomes (Westfall and Young, 1993). Even though we do observe, as expected, that significance of our point estimates decreases in some cases, all our results are robust to the adjustment.

Table 12 presents the robustness of our main ITT estimates when we use a conditional logit for cases where the outcome is binary.

Last, Table 13 presents the results of our main outcomes disaggregated by country. Columns (1) and (3) present the ITT estimates for Argentina and Colombia, respectively. Column (5) presents the estimates for the pooled sample shown previously in Table 5 for comparison. Even numbered columns show the mean and standard deviation of the control groups. Two things are worth noting in Table 13. First, we do observe that point estimates are remarkably similar between bootcamps locations. We see this as a result of a careful and of the Since we that our main design targeting program. pre-specified analysis would pool data from Argentina and Colombia together, we worked very closely with both training sites to replicate the intervention as closely as possible. Second, the fact that some point estimates lose significance when estimated separately by country proves that, dealing with the limited sample size we were dealing with, pre-specifying the pooled analysis was key in order to estimate precisely the effects of the program.

# **5 Concluding Remarks**

We evaluate the causal effects of a high-quality, intensive, part-time, computer coding training program for women on skill acquisition and employment outcomes on a sample of participants in Buenos Aires, Argentina, and in Bogotá, Colombia. We offered spots in an over-subscribed coding course to a random subset of applicants. Chosen applicants received a scholarship that covered most of the tuition costs of the course.

The bootcamps were carefully designed considering the following dimensions. First, the selection of well-established coding schools with a proven track record and experience in providing high value-added bootcamps was key to create a high return course valued by 25
the market. Second, the scholarship that covered most of the tuition costs aimed to tackle a participation constraint originated by the high costs of these courses. Third, the coding schools in both locations worked closely with gender specialists to define strategies to make bootcamps more gender inclusive, key in order to increase participation in a sample of female beneficiaries. Last, the eligibility criteria were designed to attract a sample of women with high motivation, willingness to complete the course, and high potential to benefit from it.

In this paper, we analyze the short-term effects of the program on two sets of primary outcomes of interest: skills, and employment in a technology job. Follow-up data collected between 2 and 5 months after program completion indicates that the program has a strong positive impact on skills, measured by an exam that resembles a coding test that an employer would use during an interview to an applicant for an entry-level coding job. We also find positive and significant effects of the program on the probability of getting a job in technology. Importantly, when we compare technology jobs with the rest of the jobs in our sample, we find that the former are 43 percent more likely to offer work-from home arrangements and flexible hours, and generate a job satisfaction that is 12 percent higher compared to non-technology jobs. These findings are in line with Goldin (2014). Given that the pay rate in technology jobs is similar to other jobs (at least in this short-term analysis), we interpret this increase in flexibility and job satisfaction as a net improvement in overall job quality. This is not trivial, since there exists evidence in the labor literature that documents that workers may take pay cuts when moving to a new job if that new job offers better amenities or career perspectives (Bagger et al., 2014).

The previous results refer to February 2020, which was between 2 and 5 months after program completion. Additionally, we also gathered employment information referred to May 2020. This provided us with a unique opportunity to measure whether the program effects make beneficiaries more or less resilient to the onset of the COVID-19 crisis. The evidence suggests so, especially for the sub-sample of the self-employed.

From a short-term perspective, our results are encouraging. We interpret them as proof that the program is effective at making its target population break a barrier of entry into employment in _tech_ in relatively little time. It does so by, first, designing a strong intervention that effectively tackles barriers that could prevent program participation. Then, we show that it imparts a valuable set of skills that is sought by employers in the sector, which ultimately translates into an increase in the probability of finding a job in technol26
ogy. From an external validity point of view, it has to be acknowledged that participants in this particular program consisted of a sample of women who are positively selected in terms of income, educational and labor market outcomes relative to the overall population. These results open a series of questions that will need future research. Are these positive impacts found sustained over time? How is the medium- and longer-term career progression of beneficiaries? How do participants cope with adverse effects of a long exposure to a downturn in the labor market originated by COVID-19? How effective is this type of intervention from a cost-benefit point of view? Would the impacts hold if implemented at scale? We plan to collect another round of data in a longer-term follow-up to shed light on these open questions.

27