metadata: authors: 0: author: Crepon,Bruno Jacques Jean Philippe
1: author: Premand,Patrick
count: Cote d'Ivoire
docna: 0: docna: Creating new positions ? direct and indirect
            effects of a subsidized apprenticeship program
docty: Policy Research Working Paper
subtopic: Employment and Unemployment,International Trade and Trade Rules,Skills Development and Labor Force Training,Vocational & Technical Education,Vocational Education & Technical Training
teratopic: Education,International Economics and Trade,Social Protections and Labor
abstracts: cdata!: Evaluations of employment programs
            usually focus on direct impacts on participants. Yet
            employment programs can have a range of indirect effects
            that are rarely quantified. This paper analyzes the impact
            of a subsidized apprenticeship program offering dual
            on-the-job and theoretical training in Côte d'Ivoire.
            The experiment simultaneously randomized whether
            apprenticeship positions opened by firms were filled by the
            program, and whether interested youths were assigned to a
            formal apprenticeship. This design allows for estimating
            direct impacts on youths and indirect impacts on firms
            selected to host apprentices. The analysis identifies
            whether individuals forgo other employment or training
            opportunities, and whether firms replace other workers with
            program participants. The share of youths in apprenticeships
            increased by 52.8 percentage points. This estimate accounts
            for a significant windfall effect: 26 percent of the formal
            apprentices who were placed substituted out of traditional
            apprenticeships. The inflow of apprentices into firms
            increased significantly, but also induced substitution
            effects, as firms hired 0.23 fewer traditional apprentices
            per formal apprentice placed. Overall, the net number of
            apprenticeship positions created was between 51 and 74
            percent of the number of formal apprentices placed. In the
            short term, impacts on earnings were not significant for
            youths, but firms benefited from an increase in the net
            value of work provided by apprentices.
geo_reg_key: 517191,1927320
repnme: repnme: Creating new positions ? direct and
            indirect effects of a subsidized apprenticeship program
display_title: Creating new positions direct and
            indirect effects of a subsidized apprenticeship program
listing_relative_url: /research/2018/08/30359860/creating-new-positions-direct-indirect-effects-subsidized-apprenticeship-program
topicv3: Vocational Education and Technical
            Training m1326570 1573,Vocational Education and Training m1326810 483,Trade m1326924 698,Apprenticeship m1327745 2132,On-the-Job Training m1439215 ON--2823
publishtoextweb_dt: 2018-08-20T00:00:00Z
historic_topic: Education,International Economics and Trade,Social Protections and Labor
no_of_pages: 76
author_keys: 0: author_key: 1259128
1: author_key: 921098
model_extractions: WPS8561 Policy Research Working Paper 8561

# Creating New Positions?

Direct and Indirect Effects of a Subsidized Apprenticeship Program _Bruno Crépon Patrick Premand_ Social Protection and Jobs Global Practice August 2018
Policy Research Working Paper 8561

# **Abstract**

Evaluations of employment programs usually focus on direct impacts on participants. Yet employment programs can have a range of indirect effects that are rarely quantified. This paper analyzes the impact of a subsidized apprenticeship program offering dual on-the-job and theoretical training in Côte d’Ivoire. The experiment simultaneously randomized whether apprenticeship positions opened by firms were filled by the program, and whether interested youths were assigned to a formal apprenticeship. This design allows for estimating direct impacts on youths and indirect impacts on firms selected to host apprentices. The analysis identifies whether individuals forgo other employment or training opportunities, and whether firms replace other workers with program participants. The share of youths in apprenticeships increased by 52.8 percentage points. This estimate accounts for a significant windfall effect: 26 percent of the formal apprentices who were placed substituted out of traditional apprenticeships. The inflow of apprentices into firms increased significantly, but also induced substitution effects, as firms hired 0.23 fewer traditional apprentices per formal apprentice placed. Overall, the net number of apprenticeship positions created was between 51 and 74 percent of the number of formal apprentices placed. In the short term, impacts on earnings were not significant for youths, but firms benefited from an increase in the net value of work provided by apprentices.

This paper is a product of the Social Protection and Jobs Global Practice. It is part of a larger effort by the World Bank to provide open access to its research and make a contribution to development policy discussions around the world. Policy Research Working Papers are also posted on the Web at http://www.worldbank.org/research. The authors may be contacted at Bruno.Crepon@ensae.fr and ppremand@worldbank.org.

_The Policy Research Working Paper Series disseminates the findings of work in progress to encourage the exchange of ideas about development issues. An objective of the series is to get the findings out quickly, even if the presentations are less than fully polished. The papers carry the names of the authors and should be cited accordingly. The findings, interpretations, and conclusions expressed in this paper are entirely those of the authors. They do not necessarily represent the views of the International Bank for Reconstruction and Development/World Bank and its affiliated organizations, or those of the Executive Directors of the World Bank or the governments they represent._ Produced by the Research Support Team
Creating New Positions? Direct and Indirect Effects of a Subsidized Apprenticeship Program<sup>_∗_</sup> Bruno Crépon (Crest) and Patrick Premand (World Bank) _JEL classification_ : D22, J23, J24, O12, C93.

_Keywords_ : Employment, Apprenticeship, Wage Subsidy, Training, Direct and Indirect Effects, Equilibrium Effects, Micro and Small Enterprises, Field experiment, Africa.

> _∗_ This paper is the result of a collaboration with the government of Côte d’Ivoire. We are particularly thankful to Adama Bamba, Hermann Toualy, Ismahel Abdoul Barry and Cesar Gbeugré Toassa at BCPE; Traoré Bamoudien and Ouattara Abdould Kadher at AGEFOP; as well as Hamoud Wedoud Abdel Kamil at the World Bank. Funding from the Skills, DIME i2i and Jobs Umbrella Trust Funds at the World Bank are gratefully acknowledged. We thank Sondo Eloi Somtinda for outstanding field coordination throughout the study, Joël Farronato for excellent baseline survey coordination, as well as Nicolo Tomaselli, Henriette Hanicotte and Dimanche Allo for leading follow-up data collection. Marina Tolchinsky, Matthew Olckers and Fatine Guedira provided excellent contributions to data analysis. Johanne Buba, Deon Filmer, Leonardo Iacovone and seminar participants in Cairo, ETH Zürich, Mannheim, Oxford and the World Bank provided useful comments and suggestions. All errors and omissions are our own. The findings, interpretations, and conclusions of the paper are those of the authors and do not necessarily represent the views of the World Bank or the government of Côte d’Ivoire.
# **1 Introduction**

Improving employment opportunities is one of the main policy challenges in developing countries. Governments are implementing a range of employment programs or skills development initiatives, including public works, wage subsidies, trainings or apprenticeships. Policy makers often make strong claims about the number of jobs created, the number of people trained, or the number of individuals placed through these programs. Employment programs have been widely evaluated (see Card et al. (2018) for a meta analysis), but evaluations usually only focus on impacts on participants. At the same time, it has been emphasized that employment programs can induce a range of indirect effects (Calmfors, 1994; Abbring and Heckman, 2007). While the cost-effectiveness of employment programs largely depends on the magnitude of indirect effects, they are rarely quantified.

In this paper, we report the results of the randomized experiment of a subsidized apprenticeship program offering dual on-the-job and theoretical training in Côte d’Ivoire. The experiment simultaneously randomized whether apprenticeship positions opened by firms were filled by the program, and whether interested youths were assigned to a formal apprenticeship. This is a design tailored to estimate direct impacts and _windfall_ effects among youths, as well as indirect impacts and _substitution_ effects on firms selected to host apprentices. On the one hand, participating individuals may forgo other human capital investments or employment opportunities. Windfall effects arise when some program beneficiaries would have taken a similar position absent any intervention. Such effects have been documented in studies measuring direct impacts on program participants. On the other hand, when programs seek to place individuals in firms, the total number of positions created is likely smaller than the number of individuals placed. There can be substitution effects if firms displace other workers with subsidized individuals. These substitution effects at the firm level can rarely be estimated precisely. The innovation of this paper is to simultaneously measure windfall effects among youths and substitution effects in firms. By considering both sides of the labor market, namely labor supply and earnings from individuals, as well as labor demand and profits from firms, we can estimate indirect effects and provide a more comprehensive assessment of program performance than evaluations focusing on individuals only.

The program we study offered a subsidy of 30,000 FCFA per month (approximately USD 1
54, or half the formal minimum wage), paid directly to apprentices. It included dual training, with practical on-the-job learning complemented by mentoring and theoretical courses. The apprenticeships lasted 12 or 24 months, depending on occupations. We implemented a randomization procedure simultaneously on both sides of the market, so that the study was designed to measure both windfall and substitution effects, respectively among youths and firms. Specifically, a sub-set of firms was randomly chosen to have pre-identified apprenticeship positions offered to program youths. This allows measuring how the inflow of new workers is transmitted at the firm level. We can identify workforce response and substitution effects on firms’ hiring of traditional apprentices in the private market. At the same time, we randomly assigned a subset of youths interested in apprenticeship positions to the program. We can then measure impacts on participation in apprenticeships, including windfall effects driven by youths exiting traditional apprenticeships to enter formal apprenticeships. We can also measure forgone earnings and employment for youths.

Results show that the program induces a range of indirect effects. The share of youths in apprenticeship increases by 52.8 percentage points. However, traditional apprenticeships are common in Côte d’Ivoire, and there are substantial windfall effects: 26 percent of formal apprentices placed by the program substitute out of traditional apprenticeships. The program also induces a small substitution out of formal schooling and technical and vocational training. On the side of firms, the program leads to an inflow of 1.4 formal apprentices into firms, but only 1.1 new apprentices in total when also counting traditional apprentices. The experiment thus documents substitution effects in firms: for each formal apprentice placed in a firm, 0.23 traditional apprentices are displaced.

Overall, we show that given the windfall effect among youths (0.26) and substitution effect among firms (0.23), the net number of new apprenticeship positions created ranges from 1- 0.26-0.23=0.51 to 1-max(0.26,0.23)=0.74 for each youth placed in formal apprenticeship.

The short-term cost-effectiveness of the apprenticeship scheme likewise depends on its impact on earnings for both youths and firms. We find no average short-term impacts on youths’ earnings. However, a large reorganization of activities is observed. Youths forgo employment opportunities: the share of individuals engaged in wage employment or selfemployment decreases by 13.5 and 12.9 percentage points, respectively. As a result, labor earnings decrease by 25.1 percent. Apprenticeship earnings paid by employers only com2
pensate for approximately 25.3 percent of youths’ foregone labor earnings, highlighting the large opportunity cost associated with participation in apprenticeship. However, the program subsidy fully compensates youths for losses in average labor earnings. At the same time, results suggest substantial indirect program impacts on firms. The additional inflow of formal apprentices into firms leads to an increase in labor inputs. Formal apprentices are found to be less assiduous and work fewer hours than traditional apprentices, but the aggregated value of the work provided by apprentices in firms increases. Additionally, some of the subsidies paid to youths are indirectly transferred to firms, as employers are found to reduce their own payments to formal apprentices. Overall, results show a positive significant impact on marginal profits at the firm level, as measured by the difference between the aggregated value of the work provided by apprentices and the compensation they receive from firms.

Our paper contributes to an active strand of the literature on the identification of indirect and equilibrium effects in program evaluation. In the case of employment programs, this question was first raised by Calmfors (1994) and is further discussed by Abbring and Heckman (2007). Some recent papers use large experiments or shocks to measure general equilibrium effects. For instance, Breza and Kinnan (2018) use a micro-finance crisis in India to identify the role of the aggregate demand channel, and Muralidharan et al. (2017) use the phase-in of the modernization of the large-scale NREGA program to identify both direct effects and equilibrium effects. Other papers use tailored experimental designs to identify specific types of indirect effects and mechanisms. Angelucci and De Giorgi (2009) and Bandiera et al. (2017) examine the impact of cash transfers on the non eligible population at the village level. Cunha et al. (forthcoming) compare impacts of in cash or in kind transfers to identify price effects. Other designs use a double randomization approach proposed by Moffitt (2001) to identify impacts on the eligible population that is not participating. For example, Crépon et al. (2013) study displacement effects of a job search program and Akram et al. (2017) identify a range of negative and positive spillover effects associated with emigration. The identification of indirect effects requires powerful experiments and adapted designs.<sup>1</sup> While we do not measure general equilibrium effects, our paper adds to this literature by using a new design to identify specific types of indirect effects. Our randomization procedure on the two sides of the market is well suited and powered to measure direct im-

> 1See for example Baird et al. (forthcoming) in the case of saturation designs.

3
pacts on participants as well as a range of indirect effects on firms, including substitution effects. In another recent paper, Alfonsi et al. (2017) analyze several active labor market programs in which both youth and firms are randomly assigned to treatment and control, as discussed further below.

Our paper also contributes to the literature on wage subsidies. Most empirical studies focus on the impacts of wage subsidies on employment of individual workers. Positive impacts have been documented in the short run, although results are mixed in the long run when subsidies are phased out (for reviews, see Almeida et al. (2014); Card et al. (2018)).<sup>2</sup> Berniell and de la Mata (2017) is a recent example where a 12 months wage subsidy was found to have large short and long run impacts on employment in Argentina. However, wage subsidies can affect the labor market through a wide range of indirect effects including substitution effects at the firm level and equilibrium wage effects (Layard et al., 1991; Calmfors, 1994). Lise et al. (2004) use a general equilibrium model to study the impacts of a subsidized program, and show that accounting for these effects can alter initial findings. There have been several attempts to identify indirect effects. Blundell et al. (2004) compare the population of eligible and non-eligible individuals across areas where the program has or has not been implemented and find no equilibrium effect. Some other papers have attempted to identify more specifically impacts on labor demand that could arise from substitution effects. Calmfors (1994) show that, as long as subsidized and non-subsidized workers are perfect substitutes and firms are on their labor demand curve, there is full eviction of non subsidized workers and zero overall employment effect when marginal workers are non subsidized. Most results to date come from non-experimental studies in high-income countries, with mixed results. While some studies find no substitution effects (Kangasharju, 2007; Rotger and Arendt, 2011), others find relatively large substitution effects (Dahlberg and Forslund (2005); Bishop and Montgomery (1993)). The large variation in estimates is hard to interpret in the context of non-experimental studies. Most of the existing literature also focuses on the impact of providing wage subsidies in formal firms in developed countries, with less focus on micro and small enterprises in the more informal contexts prevalent in developing countries. In an important contribution, De Mel et al. (2016) set-up a wage subsidy experiment to analyze firm labor demand in Sri Lanka. They find that informal micro-enterprises increase employment

> 2In their meta-analysis, Card et al. (2018) focus on subsidies paid to firms. In this paper, we consider a case where subsidies are paid to employees.

4
while they receive the subsidies, an effect that does not last after the end of the subsidy. Our paper adds to this literature by measuring simultaneously the impact on the demand from firms and on program participants. This enables us to estimate more precisely the net number of total positions created. Our double randomization design is powerful enough to measure both windfall and substitution effects. We find large impacts on the number of apprentices in firms but limited eviction effects, which suggests either that firms are off their labor demand curve or that subsidized and non-subsidized apprentices are imperfect substitutes.

Our paper further contributes to the literature on training and apprenticeship programs. Evidence on the effectiveness of training programs has been mixed overall (for reviews and meta-analysis, see Blattman and Ralston (2015); Kluve et al. (2016); Bertrand et al. (2013); McKenzie (2017)), though there have been recent studies with promising findings (Attanasio et al., 2011, 2017; Alfonsi et al., 2017).<sup>3</sup> Evidence on apprenticeship programs also remains very thin despite apprenticeship being one of the most common sources of training in developing countries (Teal, 2016).<sup>4</sup> On the side of youth, our paper focuses on the impact of offering participation in the program on entry into apprenticeship. As such, it contributes to the literature on the demand for training programs.<sup>5</sup> Black et al. (2003) highlight that low demand is a critical consideration in the analysis of training and employment programs. The issue has been mostly analyzed in developed countries (Babcock et al., 2012). Here, we complement the literature by considering the role of wage subsidies in the decision to enter apprenticeship in a developing country context. Importantly, it has also been argued that employers benefit from apprenticeships (e.g. Lerman (2014, 2017); Biggs et al. (1995)), but empirical identification of benefits to firms has been challenging. Recent findings in the literature tend to confirm this conjecture. Alfonsi et al. (2017) analyze vocational and on-the-job training programs and show that firms absorb a significant part of the surplus. Hardy and McCasland (2015) analyze whether a placement intervention improves the match-

> 3Attanasio et al. (2017) show that a vocational training program combining on-the-job learning and incentives for training centers to match trainees with firms had long-lasting effects in Colombia. Alfonsi et al. (2017) show that vocational training delivered by high-quality training centers and on-the-job training had positive long-lasting effects in Uganda, with vocational training outperforming on-the-job training.

> 4A few studies estimate the returns to traditional apprenticeships (e.g. Frazer (2006); Monk et al. (2008)), or short public apprenticeship schemes (Cho et al., 2013).

> 5The objective of the paper is not to document long term impacts of apprenticeship on youths, which we will study in future work based on additional data to be collected.

5
ing between potential apprentices and firms. By focusing on firms, they show that addressing inefficiencies in the screening of apprentices leads to increases in employment and profits in firms. Our analysis adds to these recent findings by also showing that the effects of a formal apprenticeship program on firms’ hiring and revenues are substantial.

Finally, our study relates to the theoretical framework developed by Acemoglu and Pischke (1999), who show that a form of monopsony power is a condition for an apprenticeship system to be viable. Firms need to be able to capture part of the surplus generated by training in order for them to deliver training, as well as for workers to believe firms will do so. This can lead to an under-provision of training, including through low participation. Our paper presents several elements of evidence related to this framework. First, it illustrates the form the monopsony power can take. In traditional apprenticeships such as those prevalent in Côte d’Ivoire, apprentices receive low remuneration during several years before they are able to exit. Second, it documents large opportunity costs of training for potential apprentices. Lastly, and importantly, our results show that a subsidized program increases participation in apprenticeship by reducing the cost of training and providing a commitment to a minimum level of training.

The paper is structured as follows. Section 2 presents the intervention, experimental design, data and estimation strategy. Section 3 documents windfall and substitution effects on youths and firms. Section 4 documents impacts on youth earnings and profits in firms. Section 5 discusses additional mechanisms. Section 6 concludes. Tables and figures are presented in the annex.

# **2 Intervention, Experimental Design, Timeline and Data**

## **2.1 Apprenticeships in Developing Economies**

Traditional apprenticeships are one of the most prevalent types of training in the developing world. In some African countries, recent data suggest that around 20 percent of youths have been an apprentice, while less than 5 percent have attended technical and vocational training (Filmer et al., 2014). The vast majority of apprentices are in traditional apprenticeships, which are one of the few sources of training accessible to the large number of youths who exited the education system without completing primary or secondary school.

6
Traditional apprenticeships are also one of the main sources of skill acquisition for informal operators. Despite its prevalence, traditional apprenticeship remains poorly understood and documented. Traditional apprenticeships are private arrangements between youths (or their families) and private sector firms. Although their form can vary, traditional apprenticeships share a range of characteristics (Walther, 2008). They take place in micro and small firms, many of which operate in the informal sector. With the help of their family, youths are often placed with master craftsmen identified through connections. A fee (in-kind or in cash) is paid for the placement. Arrangements are rarely formalized through a contract. Youth learn the trade through practical, on-the-job training by working in enterprises under the mentoring of a master craftsman, either an experienced worker or the enterprise owner. Over time, youths start being paid. Traditional apprenticeships can last many years, and often do not lead to certification, although master craftsmen typically need to grant departure to mark the completion of an apprenticeship. After completing traditional apprenticeships, youth transition either as an employee in the host firm, as a wage worker in another firm, or in self-employment. Most youth remain in the informal sector given the lack of formal wage jobs for workers with limited education.

While traditional apprenticeships have developed over time through a private, marketbased system with little public intervention, the optimality of the model has been questioned. The improvement of apprenticeship systems has become an important objective in many countries around the world (Fazio et al., 2016; OECD/ILO, 2017), particularly in West Africa (Walther, 2008; UNESCO, 2015). One of the common rationale for reforms is that policies can facilitate access to apprenticeships, while at the same time improving training quality and returns for youths. The set-up of dual apprenticeship schemes combining theoretical and practical training is often considered, modeled after institutions from high-income countries such as Germany or Switzerland. Reforms also seek to integrate and formalize apprenticeships as part of the postprimary vocational and education training system. Yet some have argued that reforms should instead consider improving quality in traditional apprenticeships that are prevalent in low-income and lower middle-income countries, rather than setting-up parallel public systems (Filmer et al., 2014). Others have made the case that resources for policies to support apprenticeships may be better spent on creating more high-productivity wage jobs (Teal, 2016).

7
The effectiveness of public interventions in the market for apprenticeships, and ultimately the rationale for reforms, depends on successfully addressing market failures faced by youths or firms. The creation of a formal apprenticeship system may induce youths to forgo other human capital investments, such as traditional apprenticeships, or employment opportunities. Beyond effects on youths, the cost-effectiveness of public apprenticeship schemes in part also depends on indirect effects, such as whether they benefit firms. For instance, it hinges on the absence of negative substitution effects on private traditional apprentices, meaning that there is unmet demand and absorptive capacity for apprentices in firms. However, impacts on firms, such as substitution effects on the hiring of other apprentices, are rarely quantified. The literature on the benefits and costs of apprenticeships is extremely thin, and mostly concentrated in a few high-income countries such as Germany or Switzerland (for a review, see Lerman (2014, 2017)). The lack of evidence in developing countries is particularly striking given the prevalence of apprenticeship as one of the most common types of training, its mostly private nature, and the importance it is given in skills development or youth employment strategies. Most existing studies focus on access and returns for youths, with little evidence on indirect effects on firms, and little distinction between traditional and formal apprenticeships.

# **2.2 The Côte d’Ivoire Formal Apprenticeship Program**

After steady economic development through the mid-1990s, Côte d’Ivoire entered a period of conflict, punctuated by a post-electoral crisis in 2010-11. Stability returned after the institution of a new government in 2011, and economic performance has improved since then. A range of public investments and programs were launched in 2011. They included an emergency youth employment and skills development project (PEJEDEC), which had an objective to improve access to temporary employment and skills development opportunities for young men and women in Côte d’Ivoire.<sup>6</sup> Among other interventions, the project included an apprenticeship component.<sup>7</sup> The PEJEDEC apprenticeship component is overseen by the office coordinating employ-

> 6PEJEDEC: Projet Emploi Jeune et Développement des Compétences (www.pejedec.org).

> 7See Bertrand et al. (2017) for evidence on the cost-effectiveness of a public works program supported by the same project.

8
ment programs (BCP-Emploi)<sup>8</sup> , with the national training agency as implementing agency (AGEFOP).<sup>9</sup> The program puts in place a formal apprenticeship scheme lasting 12 or 24 months, depending on occupations. The program initially aimed to cover 3,000 youths, and is in the process of being expanded to approximately 10,000 youths. Low-skilled youths between 18 and 24 years old are placed in firms, where they receive on-the-job training under the supervision of a master craftsman, either the enterprise owner or an experienced employee. Youths sign a contract with the implementation agency (AGEFOP) and are paid a monthly subsidy of 30,000 FCFA (approximately USD 54, or half the formal minimum wage), which is aimed to cover meals and transport costs. They receive an insurance coverage and work equipment. The apprenticeship is dual, since on-the-job practical training is complemented by theoretical training (approximately 180 hours per year) tailored to the needs of apprentices and delivered by local training institutions. Apprentices are also mentored by AGEFOP apprenticeship counselors, who regularly visit master craftsmen and apprentices, and have the authority to suspend subsidies in case there are issues with youths’ participation or performance. Formal apprenticeships end with an assessment of youths’ skills, leading to certification. Firms are not compensated for taking on apprentices, though they do receive a small toolkit of material to facilitate practical learning. Moreover, employers commit not to request the payment of tuition fees at the start of the apprenticeship, in contrast to the traditional apprenticeship model in West Africa (Walther, 2008).

The average program cost is estimated at FCFA 1,135,030 (approximately USD 2,045) per youth for a 24 months apprenticeship. This includes FCFA 720,000 (or USD 1,297) for subsidies for youths, FCFA 330,000 (or USD 595) of other direct costs (toolkit, theoretical training, equipment,...), and FCFA 85,030 (approximately USD 153) for indirect costs (selection, counseling, and so forth).

# **2.3 Enrollment Process and Experimental Design**

One of the main objectives to embed an experiment in the Côte d’Ivoire formal apprenticeship program was to measure simultaneously windfall effects among youth and indirect effects in firms. This requires a specific design that randomly assigns both youth and firms to treatment

> 8BCP-Emploi: Bureau de Coordinnation des Programmes d’Emploi.

> 9AGEFOP: Agence de la Formation Professionnelle.

9
and control groups. In this section, we present the experimental protocol and discuss how it addresses potential interference.

The program was implemented in 7 urban areas in the interior of the country.<sup>10</sup> In each locality, AGEFOP worked with private sector organizations (such as chambers of commerce or trade associations) to identify firms interested in hosting formal apprentices. For each firm, the number of available apprenticeship positions was collected. AGEFOP staff then systematically visited all firms to explain the program, to check each firm’s ability to train apprentices and confirm the number of apprenticeship positions they could offer. A baseline firm survey was implemented right after the collection of apprenticeship positions. Once all positions were identified in a given locality, they were grouped and advertised by trade.<sup>11</sup> Youths between 18 and 24 years old were then invited to visit a central location in each locality to apply for apprenticeship positions in available trades.<sup>12</sup> They filled an application form and indicated the trade they were interested in. The program targeted low-skilled youths, but did require an ability to read and write to ensure youths could participate in the theoretical training, de facto implying that youths had at least a few years of schooling. Youths who met basic eligibility criteria were invited to an interview with AGEFOP apprenticeship counselors. The interview sought to confirm youth motivation for doing an apprenticeship as well as their choice of trade. A baseline survey was implemented among all youth who successfully passed the interview.<sup>13</sup> This registration process led to the same number of youths eligible and motivated than open positions, in each locality and in each

> 10These included Man (35% of youth in the sample), Daoukro (15%), Gagnoa (14%), Divo (12%), Bouaké (12%), Adzopé (7%) and Mankono (5%). It was planned that Abidjan would also participate in the experiment, but demand for apprenticeship positions among youth was limited and there was not enough oversubscription, so that Abidjan had to be dropped from the sample. The program was launched between July 2014 (Adzopé), August 2014 (Daoukro, Gagnoa, Man and Mankono), September 2014 (Divo) and October 2014 (Bouaké). A target number of youth to include in the program was set for each of these localities based on the estimated number of available apprenticeship positions and other considerations.

> 11Throughout the paper, we make a distinction between "sectors" and "trades". Sectors refer to the activity of the firm and "trades" refer to jobs taught to youth. The two concepts are often the same, but in some cases firms in a given sector are active in several trades. A good example is the garage sector, which includes apprenticeship positions in several trades: coach builder, car mechanic, car electrician, and car painter.

> 12The most popular trades included car or motor mechanic (21% of positions), metalworker, boilermaker, welder (14%), bricklayer, painter, plumber (11%), carpenter (9%), car electrician (9%), electrician (8%), coach-builder (8%), repairman for fridges and freezers (7%).

13Despite the efforts made to advertise the program, it was not possible to find enough youths interested in some trades in some localities. In such cases, a rationing occurred at the firm level: the number of positions to be filled was reduced proportionally, while ensuring as much as possible that firms would keep at least one open position. In a few cases, this was not possible and some firms had to be randomly excluded, even though they had been initially registered and surveyed.

10
# trade.

In each locality, following the interviews, a double-sided randomization protocol was implemented. The procedure was the following: firms were paired according to the number of positions they opened per trade, and within each pair a firm was assigned to treatment and another to control. The reason for implementing this pairing procedure, instead of a theoretically more appealing stratification by trade, is that some firms opened positions in different trades (see footnote 11). Once the firm randomization was implemented, the number of open positions in treatment firms was counted by trade. This gave the exact number of youths to select. The second step of the randomization was then implemented, randomly assigning youth to treatment by trade. We assigned the exact same number of youth to treatment as the number of open positions to fill. As a result, the probability of youth assignment to treatment is trade-specific. On the side of youth, the experiment is thus stratified by locality (since the randomization procedure was implemented separately in each locality) and by trade. Since assignment probabilities are strata-specific, we include strata-specific weights in the specification used to estimate impacts on youth, as discussed further below. Figure A1 in appendix A1 shows the distribution of this ratio by stratum, showing as expected a strong concentration around 0.5.<sup>14</sup> Once apprenticeship positions to be filled were selected, and youths were selected in each locality, AGEFOP counsellors matched selected youths to selected firms with open positions in the same trade. The matching took place based on criteria such as distance between the firm and youth residence. Once assigned to firms, youth passed a medical visit and were invited to sign a contract and start their apprenticeship.<sup>15</sup> Across the 7 localities covered by the study, 731 firms offered apprenticeship positions and 1,832 young applicants were eligible and passed the motivation interview. Approximately half the firms (361), were randomly selected to host program apprentices. 911 eligible and motivated youths were assigned to the program and 921 to the control group. Most firms

> 14As can be seen from the figure, there are a few cases in which the assignment ratio is either 0 or 1. The appendix explains how we deal with these cases.

> 15This is a demanding experimental protocol requiring a lot of specific actions and close coordination with the implementation agency in a short period of time. We had a team of three highly skilled research associates based in the field, as well as a full data collection team implementing baseline surveys. Once the experimental protocol was implemented in a given locality, a detailed summary report was written to list all the specific implementation aspects. For example, the report registered the initial number of positions offered in each trade, the number of firms involved, as well as any rationing that occurred and the number and identity of any firm randomly excluded from the experimental protocol.

11
offered several positions, and on average treatment firms were assigned 2.52 apprentices.

Importantly, substitution effects at the firm level and windfall effects at the youth level imply that labor market flows are affected by the intervention. Some youth not registered in the experiment (as well as youth assigned to the control group) could potentially be displaced and have to find a different position than the one they would have obtained absent the program. Similarly, in presence of windfall effects, some treated youth hired in treatment firms may have left open positions in firms where they would have been working absent the program. These effects are both of primary interest of the study, but can also lead to interference in the measurement of program impacts.

We summarize here how our experimental protocol helps address the challenges of measuring windfall and substitution effects. Appendix A1 provides a more detailed discussion. First, youth and firms have to be statistically identical in the treatment and control groups. Proper implementation of the randomization protocol ensures this. In addition, our experimental design is implemented at the level of micro-markets, defined as a given trade in a given locality. In this context, a second requirement is that youth and firms in the control group have the same opportunities in their micro-market as what they would have had absent the program (this is described as assumption A(1) in appendix A1). The experimental design seeks to achieve this property by maintaining a similar share of control youth and control positions in each micro-market. A third important dimension is that windfall and substitution effects can potentially change the flows of non-registered youth in the apprenticeship market, as well as the flows of youth towards non-registered firms. An additional requirement is thus that the corresponding match probabilities are not affected by the experiment (this is described as assumption A(2) in appendix A1). For this property to hold, the shares of treated youths and firms in the apprentice market need to remain small. Using information from a recent national employment survey and population census, we can show that this share is estimated to be 5% (see discussion in appendix A1 and Table A2). Overall, the fact that these three requirements are met mitigates risks of interference.

# **2.4 Data and Estimation Strategy**

The program was rolled out sequentially, locality after locality. As explained in the previous section, baseline data was collected in each locality as part of the enrollment process. Specif12
ically, after the apprenticeship positions offered by firms were validated by program staff, a comprehensive baseline survey was implemented in each firm with confirmed positions. Separately, baseline data were collected among youth deemed eligible after they successfully passed the motivation interview. Baseline data collection among firms and youth took place in each locality before the randomization was performed. The baseline and enrollment phase took place between July 2014 and October 2014. The selection of firms and youth took place shortly after, and placements were mostly completed by January 2015.

The follow-up survey for the impact evaluation took place between March 2016 and June 2016. It was collected on average 20 months after the start of the program.<sup>16</sup> Since most apprenticeships last 24 months, results based on the follow-up survey should be interpreted as providing short-term impacts while apprentices are still in the program.<sup>17</sup> Substantial efforts were made to minimize attrition during the follow-up survey.<sup>18</sup> As a result, 1,661 youth were surveyed, implying an attrition rate of 9.3% (or 171 youth, 84 in Treatment and 87 in Control). Similarly 674 firms were surveyed, leading to 7.8% attrition (or 57 firms, 26 in Treatment and 31 in Control).<sup>19</sup> Tables A3 and A4 present baseline characteristics and balance checks for youth, respectively firms. Both tables have the same structure, the left panel is devoted to the analysis of baseline data (including on the last row the share of youth or firms with available data). The right panel presents baseline characteristics of follow-up respondents and related balance checks.

Table A3 shows that youth interested in formal apprenticeships are 20.7 years old on average, and mostly men (87 percent). They have some (but limited) education, as 63 percent have completed primary school and 17 percent lower secondary school. 45 percent of applicants aspire to a wage job, and 54 percent to become self-employed. There are few significant differences between the treatment and control group of youth, who are largely

> 16Figure A2 documents precisely the timing of surveys as a function of the randomization date.

> 17 754 of the 914 youths in the treatment group (or 82%) were in trades where the apprenticeship contract lasted 24 months.

> 18An unfortunate IT issue with the online server used for electronic data collection led to the loss of baseline data for 26% of youths (475 youth, 250 in Treatment and 225 in Control) and 5% of firms (37 firms, 18 in Treatment and 19 in Control). The problem was concentrated in two localities. The loss of some baseline data limited availability of contact information to track youth (and firms) at follow-up. This contributed to a lower response rate among youths in localities where IT issues occurred.

> 19It is worth to note that part of this attrition is due to firm closure. We designed a specific data collection instrument for registered employers whose firm had closed by the time of the follow-up survey. 12 cases were identified. "True" attrition is limited to 6.2% (45 firms, 23 in Treatment and 22 in Control).

13
comparable and well-balanced at baseline. As can be seen from the table, the share of available baseline data is not perfectly balanced (see footnote 18), but the response rate at follow-up is well-balanced, which is what matters most since it is the basis for empirical estimation.

Table A4 highlights that most firms offering apprenticeship positions are informal micro and small enterprises. 84 percent have no formal legal status and 68 percent do not keep books. Firms have 6.3 permanent employees on average (counting the owner), of which 3.3 are apprentices. Traditional apprentices therefore constitute more than half the workforce in these micro and small firms. Traditional apprentices are mostly hired through private channels, 82 percent based on a request from their parents. About half of the apprentices in firms at baseline pay fees to the master craftsmen. These traditional apprenticeships are expected to last six years on average. Table A4 documents that the experiment led to good baseline balance between the treatment and control firms: the few significant differences are marginal and of small magnitude.

The follow-up survey was collected by phone for youth in the treatment and control groups. The most important variables for the analysis are youth activities at the moment of the survey, as well as hours and earnings in those activities. The survey contains a detailed employment module covering primary and secondary activities. We can thus describe the portfolio of youth activities, distinguishing between occupations as apprentice (formal or traditional), wage worker and self-employed. Other important variables relate to youth human capital investments, including participation in apprenticeship (formal or traditional), vocational training and schooling.

The follow-up firm survey in the treatment and control groups took place in person. It collected data on the characteristics of the firm, its workforce, as well as its sales and profits. It also compiled a listing of all apprentices who entered or left the firm since the start of the experiment (i.e. on or after the randomization date in each locality), and collected additional information on each of these apprentices, both from enterprise owners and from apprentices themselves. This employer-employee type of data enables us to accurately measure the flows of apprentices in and out of firms, as well as their contribution to firm activity. For example, we are able to compute the number of apprentices working in firms at the moment of the survey, but also various interesting flows: the number of apprentices who entered firms since 14
randomization, and among them those who left firms and those still in firms. We can measure all these variables separately for formal and traditional apprentices. The survey also asks about the number of days worked in the last seven days and the number of hours worked in the last business day for each apprentice. In order to measure apprentices’ contribution to firm activity, we asked employers about the amount he would have had to pay had he hired a casual worker to perform the same tasks. This in turn allows us to compute a value for the work performed by each apprentice. We also asked employers about the compensation paid to each apprentice. These measures can be aggregated at the firm level.<sup>20</sup> We also collected several measures of sales and profits, addressing concerns about the measurement of these variables raised by De Mel et al. (2009). Following the procedure recommended in De Mel et al. (2009), we obtained direct measures of total profits and revenues. We then asked the firm owner to recall all sales over the last month as well as related expenses. On that basis, we collected another (updated) measure of total profits and revenues. Moreover, we implemented near-systematic back-checks of key variables, including sales and profit. Thus, for most firms in the sample, we have six measures of sales and profits.

Given the double-sided randomization protocol, intent-to-treat (ITT) program impacts on firms can be estimated by comparing outcomes between firms assigned to treatment (i.e. where formal apprentices were assigned by the program to fill open positions), and firms assigned to control (i.e. where open apprenticeship positions were not filled by the program). The ITT analysis at the firm level is performed using OLS regressions with the 667 firm-level observations at follow-up: We compute robust standard errors. In this equation _T_ is the assignment to treatment variable, _v_ stands for the locality and _s_ for the sector. Sectors are a broader concept than trade (see footnote 11).

In parallel, intent-to-treat program impacts on youth can be estimated by comparing

> 20We made a distinction between various forms of compensation. Employers usually provide meals and cover expenses such as transportation and clothes. They also provide some money for the work done by youth in order to "motivate" them. We measure each of these payments and aggregate them by youth and by firm to get a total wage bill for apprentices.

15
outcomes between youth assigned to treatment (i.e. offered a formal apprenticeship position in a treatment firm), and control youths. We have, however, to account for the fact that youth were assigned to treatment and control with probabilities that were specific to each trade in each locality, producing a set of corresponding strata _St_ . We compute the empirical assignment rate in each stratum and estimate inversely propensity weighted regressions.<sup>21</sup> To obtain accuracy gains from stratification, we run an inversely propensity weighted regression with strata dummies on the 1,661 youth observations: We compute robust standard errors.<sup>22</sup>

# **3 Windfall and Substitution for Youth and Firms**

## **3.1 Youth Entry into Apprenticeship and Windfall Effects**

Table 1 presents ITT estimates of human capital investments among youth, covering the experiment period between randomization and the follow-up survey. We consider human capital investments in the form of schooling and training. As part of training, we distinguish between technical and vocational training (TVET) and apprenticeship. Participation in apprenticeship is further decomposed between traditional apprenticeships, i.e. private arrangements that exist independently of the program, and formal apprenticeships of the type promoted by the program.

Table 1 clearly shows, as expected, that the program leads to a large increase in participation in formal apprenticeships: the share of youth in formal apprenticeship is 71.2 percentage points larger in the treatment group than in the control group. A very low, but non-zero participation in formal apprenticeships is observed in the control group. Overall, 75 percent of youths in the treatment group participate in formal apprenticeships, which is highly

> 21The empirical assignment rates are defined on the sample used in the regression.

> 22There are 1,676 observations for which we were able to collect follow-up data. However, there are 15 youth observations for which the empirical assignment probability within their stratum is either 0 or 1, and these are discarded from the youth regressions.

16
consistent with program take-up measures based on process evaluation or administrative data.<sup>23</sup> Importantly, Table 1 documents substantial windfall effects: participation in formal apprenticeship is in part explained by substitution out of other forms of human capital investments, such as youth in the treatment group substituting out of traditional apprenticeships. 22.5 percent of youth in the control group participate in traditional apprenticeships, a proportion that is reduced by 18.5 percentage points in the treatment group. The windfall effect is of substantial magnitude. It can also be expressed in relative terms, as the share of youths exiting traditional apprenticeships (18.5 percentage points) relatively to the share of youths entering formal apprenticeships (71.2 percentage points), or 26 percent. In other words, for each formal apprentice placed, 0.26 youths substituted out of traditional apprenticeship. Overall, the net program impact on the share of youth in any form of apprenticeship over the course of the experiment is 52.8 percentage points.

The program also induced a small substitution out of other forms of human capital investments such as schooling or technical and vocational training (TVET). The participation in TVET in the control group is quite low (7.2 percent), but the program has a relatively large impact in reducing this proportion by 5.7 percentage points.<sup>24</sup> Last, the program also has a significant and negative impact on youths attending school. 20.5 percent of youth in the control group report going to school, and this proportion is reduced by 5.7 percentage points in the treatment group.

Despite these indirect effects, a large proportion of youth makes no human capital investment of any form in the control group (51.4 percent). Offering participation in the program sharply reduces this proportion by 36.3 percentage points, approximately half the program take-up. In the end, only 12.3 percent of youth in the treatment group do not make any form of human capital investment.

So far, we have focused on impacts on entry into apprenticeship between the baseline and

> 23Appendix A2 and Table A5 provide additional information on take-up measures from other data sources.

> 24To build human capital investment variables, we use a question in the follow-up survey about participation in apprenticeship and TVET, as well as a separate question about youth participation in public apprenticeship programs. We classify as participation in formal apprenticeship those who are either in apprenticeship or TVET and say they were registered in a public apprenticeship program. Although not perfect, and subject to some recall bias, this is the classification that is most consistent with information from the process evaluation survey, for the subsample of treated youth that can be matched in both surveys. See Appendix A2 for a more detailed discussion.

17
follow-up surveys, which captures inflow into apprenticeships. We can also analyze impacts on participation in apprenticeship at the time of the follow-up survey. In Table 1, the right part of the lower panel presents ITT estimates of impacts on youth participation in apprenticeship at the time of the follow-up survey (i.e. approximately 20 months after the start of the program). The data is obtained from a survey module on youth occupations, which includes apprenticeship as an option. The information covers all types of apprenticeships, including traditional and formal apprenticeships. The left part of the lower panel of Table 1 presents ITT estimates for the share of youth who started apprenticeship but dropped-out by the time of the follow-up survey. Those estimates (labeled as _d_ in the table) are simply obtained as the difference between estimates of entry in apprenticeship since the start of the experiment (labeled as _e_ ) and estimates of youth still in apprenticeship at the moment of the survey (labeled as _c_ ).

Drop-out is a common issue in many employment or training programs. It has been shown to be important in apprenticeship as well (e.g. Cho et al. (2013)). We can compute drop-out rates from Table 1. Drop-out is estimated as 0.222/0.712=31.2% in formal apprenticeships and 0.060/0.185=32.5% in traditional apprenticeships.<sup>25</sup> These results show that drop-out is important, but also that it is not an issue specific to formal apprenticeships as the drop-out rate in traditional apprenticeship is close.<sup>26</sup> Although it is reassuring that drop-out is not higher in formal apprenticeship, these results show that the program did not reduce the prevalence of drop-out compared to traditional apprenticeships.

Results on participation in apprenticeship at the time of the follow-up survey confirm previous findings on increased participation but also significant windfall effects. The share of youth in formal apprenticeship increases by 49 percentage points. However, the share of youth in any type of apprenticeship increases by only 36.5 percentage points. The difference between the two estimates, 12.5 percentage points, provides another estimate of the windfall effect, namely the share of youth who substituted out of traditional apprenticeships to be in formal apprenticeships at the time of the follow-up survey. Expressed relatively to the share of formal apprentices placed, the windfall effect is again 26 percent (12.5/49), which is the

> 25We could also obtain these estimates by running a regression of dropout status on entry into formal or traditional apprenticeship using the treatment assignment variable as an instrument.

> 26These numbers overestimate drop-out in the treatment group because a small share of formal apprenticeships only lasted 12 months and these contracts were over at the moment of the follow-up survey. See footnote 17.

18
same as the estimate over the period between baseline and follow-up.

These results show that some youth decided to forgo undertaking traditional apprenticeship to participate in formal apprenticeship, again pointing to substantial windfall effects. Overall, 54.4 percent of youth in the treatment group are in apprenticeship at follow-up, compared to 17.9 percent of youth in the control group. Most youth in the control group are in traditional apprenticeship (16.1 percent out of 17.9 percent), while most youth in the treatment group are in formal apprenticeship (50.8 percent out of 54.4 percent). The fact that youth in the control group were able to find traditional apprenticeship positions by themselves through the private market, and that some formal apprentices substituted out of traditional positions, thus creates a windfall effect that mitigates the net program impacts on the share of youth in apprenticeship.<sup>27</sup>

# **3.2 Intake of Apprentices and Substitution Effects in Firms**

We now turn to the other core research question of whether the program induced indirect employment effects among firms. The analysis is based on a survey module asking employers to list all apprentices who have worked in the firm over the course of the experiment, including those who left the firm since the randomization. We can thus measure flows of apprentices in and out of firms between the start of the experiment and the follow-up survey. We can also distinguish between formal and traditional apprentices. Table 2 (upper panel) documents the impact of the program on the flow of apprentices into firms since the date of the randomization. The program led to an increase in the total number of apprentices that entered by 1.08 new apprentice per firm over the course of the experiment. Yet the total number of youth who entered formal apprenticeship in these firms increased by 1.398. As such, the program induces a significant substitution effect: fewer traditional apprentices were hired as additional formal apprentices entered firms. Substitution effects were of moderate magnitude, however, with (1.398-1.08)/1.398 = 0.227 informal apprentice substituted per formal apprentice placed.

The net increase in formal apprentices in treatment firms can be contrasted with the number of apprenticeship positions offered by these firms. On average, firms offered 2.51

> 27Additional results are presented in Table A7, which documents the time evolution of participation in apprenticeship using information from a retrospective calendar of occupations covering the four quarters before the follow-up survey.

19
apprenticeship positions.<sup>28</sup> A substantial share of these positions were not filled, and the placement ratio is relatively weak: only 1.398/2.51 = 55.5% of positions offered by firms were effectively filled. These patterns are consistent with substantial drop-out among selected youths. This also suggests that firms face challenges in attracting youths, even despite the program subsidy.

Figure 1 provides additional information about the impact of the program on the inflow of apprentices into firms. The figure shows monthly inflows of apprentices in treated and control firms by date (with zero being the randomization date). The figure makes a distinction between inflows of formal apprentices and traditional apprentices in treated firms. The figure clearly shows a spike of entry of formal apprentices in treated firms shortly after randomization. It also shows that flows of traditional apprentices were weakly affected by the program.

Table 2 (intermediate panel) documents the impact of the program on exit of apprentices out of firms. Exits from apprenticeship are substantial. Out of 1.08 apprentices who entered firm due to the intervention, 0.467 apprentices had exited by the time of the follow-up survey. This amounts to a 43.2% drop-out rate. As discussed in the previous section, drop-out is a pervasive aspect of apprenticeship. While it is large, it is not higher in the treatment group than in the control group.<sup>29</sup> Table 2 (lower panel) documents net program impacts on the number of apprentices who entered since randomization and are still in firms at the time of the follow-up survey. Results can simply be deduced as the difference between the top two panels. They show that, 20 months after the launch of the program, there are 0.613 total apprentices per firm that entered since the date of randomization, and 0.787 additional formal apprentices per firm. Substitution effects can also be measured at the time of the follow-up survey. They are again positive and significant, but remain of relatively moderate magnitude, with (0.7870.613)/0.787 = 0.221 traditional apprentice substituted per formal apprentice placed. This estimate is very close to the one obtained looking at inflows between the baseline and followup surveys.<sup>30</sup>

> 28There are 911 youth assigned to 361 firms in the full treated sample (see section 2.3), resulting in an average of 2.52 per firm. In the regression sample, 864 youths were assigned to 334 firms resulting in an average of 2.51 per firm.

> 29If we consider youth who entered firms within 6 months after randomization, we find drop-out rates of 43.3% in the treatment group and 47.1% in the control group.

> 30As can be seen in the top two panels of Table 2, there is also some non-compliance with the experimental

20
The follow-up firm survey also provides information about the total workforce in firms (including apprentices and other types of employees). Results are presented in appendix Table A8. While there is a significant impact on the inflow of youth that entered firms since the beginning of the experiment, there is no significant impact on the overall number of apprentices in firms at follow-up. The estimated impact is 0.464 with a standard error of 0.362. The impact on flows are not large enough to affect stocks significantly, which may be due in part to large standard errors in the stock variables.<sup>31</sup> Consistent with the lack of significant impact on the total number of apprentices in firms, the program does not have an impact on the overall workforce in firms. No significant impact is found on the number of interns or occasional workers either.

Table A8 shows that the program reduced the share of firms without any apprentice. Some firms that opened apprenticeship positions had no apprentice. In the control group, 20.3% of firms have no apprentice at the time of the follow-up survey. This proportion is significantly lower by 5.4 percentage points in the treatment group, but there are still 14.9 percent of firms without apprentice despite these firms having offered positions that were selected to be filled by the program.

In terms of measurement, the study shows the importance of carefully defining outcome measures. If we had implemented a simple survey only asking about the number of employees and apprentices in firms, we would have concluded that there was no significant impact on employment. However, this stock measure would have missed important impacts of the program. Indeed, the impact on stock is the combination of impact on inflows and outflows (see footnote 31). In addition, the flows between the start of the experiment and follow-up combine both entry and dropouts. With an employer-employee survey, data on the dates of entry and exit for each apprentice enable us to build a more precise set of measures of both entry and exit, providing a richer understanding of indirect impacts on firms.

protocol. The average number of formal apprentices who entered since the start of the experiment per control group firm is 0.188. As can also be seen from the table, this imperfect compliance did not last long: most (0.13) formal apprentices in control firms had left by the time of the follow-up survey. The non-compliance was highly concentrated in two out of the seven localities in which the program was implemented.

> 31For completeness, the impact of 0.464 on the stock of apprentice combines the previous impact of 0.613 on entry by follow-up with the impact on total number of apprentices who were in the firm before the randomization and are still in the firm at the moment of the survey, -0.154 (with a standard error of 0.239).

21
# **3.3 Net Impact on Apprenticeship Positions Created**

So far, we have discussed ITT impacts of offering youth to enter formal apprenticeships, and on assigning formal apprentices to firms with open positions. Results show that there are windfall effects on youth as well as substitution effects on firms. These effects imply that the net number of positions created by the program is smaller than the number of formal apprentices placed. In this section, we discuss more precisely what is the overall impact of the intervention by discussing how both effects combine, and by quantifying the net impact on the number of apprenticeship positions created.

We first turn ITT estimates into LATE estimates that represent impacts per youth entering formal apprenticeships. On the youth side, we estimate: where _ai_ stands for having started an apprenticeship since the beginning of the experiment and _fi_ for starting a formal apprenticeship. We estimate this equation using the treatment assignment variable as an instrument. In this equation, (1 _− ω_ ) represents the proportion of formal apprentices who entered apprenticeship due to the program. Thus, _ω_ measures the proportion of formal apprentices who would have started an apprenticeship anyway absent the program. Said differently: for each youth starting a formal apprenticeship, there are _ω_ fewer youth starting a traditional apprenticeship.

We also analyze the impact of entries into formal apprenticeships on the total number of entries of apprentices in firms. We estimate: where _ei_ is the total number of youth entering apprenticeship in firms and _e_<sup>_f_</sup> _i_<sup>isthetotal</sup> number of formal apprentices hired by firms. In this equation, (1 _− σ_ ) measures the number of youth entering the firm per formal apprentice placed. Thus, for each formal apprentice entering the firm, there is _σ_ less youth entering as a traditional apprentice. This equation is estimated using the treatment assignment variable as an instrument.

Table 3 presents the results. The fist two columns present the reduced form, which are 22
the ITT estimates presented above. The third column presents IV estimates for (1- _ω_ ) and (1- _σ_ ), which are simply the ratio of the first two columns. The last column provides the estimated substitution and windfall parameters _σ_ and _ω_ . As can be seen from the table, the estimated value of the windfall parameter among youths is 0.259, with a standard error of 0.022. On the firm side, there are 0.773 youths entering firms per formal apprentice placed, thus leading to an estimated substitution parameter of 0.227 with a standard error of 0.128.

One important question is how to combine these estimates to derive the net number of apprenticeship positions created by the program per apprentice placed. The windfall effect _ω_ can be interpreted as the proportion of youth who would have entered traditional apprenticeships absent the program. Consistent with the fact that the size of the experiment is small compared to the size of local apprenticeship markets (see Section 2.3 and Appendix A1), traditional apprenticeship positions which youth did not enter are located in the group of firms not registered in the experiment. Thus, we consider that, for each program youth, there are _ω_ additional positions to fill in the set of non-registered firms. Similarly, the substitution effect _σ_ is interpreted as the number of positions that firms would have filled with traditional apprentices, per program youth placed, had the program not been implemented. Again since the size of the experiment is small, those traditional apprentices would have been hired from the set of non-registered youth. For each entry of one additional program youth in a treated firm, there are thus _σ_ additional non-registered youth in search for a position. Our understanding of the flows created by the implementation of the experiment is thus quite simple: there are _σ_ additional positions to fill in the group of non-registered firms, and there is _ω_ additional youth in search of apprenticeship positions. Figure 2 summarizes those flows. Part of the additional youths searching for traditional apprenticeships may match with firms with open positions ( _me_ ). The total number of lost traditional apprenticeship position _le_ due to the implementation of the experiment is thus _le_ = _σ_ + _ω − me_ for one additional formal apprentice placed. Because we do not follow non-registered youth and firms, we are unable to measure the number of matches _me_ . However, it can easily be bounded: 0 _< me < min_ ( _σ, ω_ ). As a result, the number of lost apprenticeship positions can be bounded by We directly derive the bounds for the net number of apprenticeship positions created per 23
additional apprentice placed _ne_ as Overall, the net number of apprenticeship positions created by the program was between 51 and 74 percent of the number of formal apprentices placed.

# **4 Earnings for Youths and Firms**

So far, we have focused on results on youth participation in apprenticeships, and indirect effects related to the number of new apprenticeship positions in firms. The discussion has shed light on the presence and magnitude of windfall and substitution effects, respectively among youth and firms. We now turn to analyzing the short-term impacts of the apprenticeship program on earnings for both youth and firms. This provides additional information on opportunity costs from participation in apprenticeship among youth, as well as potential indirect to

## **4.1 Youth Employment and Earnings**

We first analyze short-term impacts of the program on youth employment, activities and earnings. Results show that there are substantial opportunity costs for youth to participate in apprenticeships. Table 4 documents ITT estimates (equation 2) for employment, hours worked and earnings by type of employment.

The upper panel of the table presents results on youth activities. It shows that youth in the control group are mostly active, as 91 percent are engaged in some economic activity. Moreover, the average number of activities in the control group is larger than one, indicating that some youths have several activities. In this section, the program only induces a small increase in participation in economic activity (by 3.4 percentage point), and a small increase in the average number of activities (by 0.05). However, the program induces youth to reorganize their portfolio of activities and forgo some employment opportunities. Specifically, individuals in the treatment group are less likely to hold wage jobs (by 13.5 percentage point) or to be self-employed (by 12.9 percentage point), and more likely to become apprentices 24
(by 36.5 percentage points).

The intermediate panel of the table presents results about hours worked and shows similar effects. Total hours of work only marginally increase (by 3.7 hours per week). The increase in hours worked as apprentices (+18.2 hours per week) is offset by a decrease in hours worked in wage employment (-6.5 hours per week) and in self-employment (-7.7 hours per week). These results are broadly consistent with the overall employment situation in Côte d’Ivoire, where unemployment is relatively low, and most youth are engaged in some type of employment, often in agriculture, non-agricultural self-employment or informal wage jobs (Christiaensen and Premand, 2017). In this context, Bertrand et al. (2017) also find that the impacts of a public works program on employment mostly take the form of a reorganization of economic activities, as opposed to an increase in overall employment or activity rates.

The third panel of the table presents estimates of program impact on total earnings, earnings by source of employment, and non-labor earnings. Overall, the program has no short-term impacts on average earnings among youth. Results show that labor earnings decrease by FCFA 10 494 (or 25 percent), while non-labor earnings increase by 10 213 FCFA (or 135 percent). The decrease in employment earnings is driven by a decrease in earnings in wage employment (- FCFA 6 414) and self-employment (-FCFA 6 381), which is only partly offset by an increase in apprenticeship earnings paid by employers (+ FCFA 3 238). The program subsidy, which is paid by the implementing agency (and not the firm), is included in non-labor income. Non labor earnings increase by FCFA 10 213 in the treatment group, driven by the subsidy. As such, it is only after accounting for the program subsidy that forgone labor earnings are fully compensated. Overall, although the total number of hours worked increases, employment earnings decrease, and total earnings remain stable.

The bottom panel of the table presents average hourly earnings in the different occupations across the treatment and control groups. Those average hourly earnings are simply obtained by dividing earnings in a given occupation by the number of hours in that occupation. Comparisons of hourly earnings between activities and across groups are informative, although they should not be interpreted as causal, since there is selection into different occupations. The table shows that, in the control group, youths involved in apprenticeship earn on average FCFA 628 per hour from their employers, far lower than hourly earnings in wage employment or self-employment (respectively FCFA 1030 and FCFA 1082). This suggests 25
large opportunity costs of apprenticeships. Interestingly, average hourly labor earnings of apprentices in the treatment group (FCFA 310) is far lower than average hourly earnings of apprentices in the control group (FCFA 628). However, accounting for the program subsidy, the average hourly earnings of apprentices in the treatment group (FCFA 706) is larger, although it remains lower than earnings in wage employment or self-employment. This illustrates how the subsidy changes the structure of payments made by employers to apprentices: the provision of the subsidy leads to a behavioral response from employers, who in turn pay apprentices less.<sup>32</sup> Overall, results show that the opportunity costs of participating in apprenticeship are quite large. Individuals are foregoing earnings in wage jobs and in self-employment, and the program subsidy contributes to balancing the financial costs of undertaking apprenticeships.

The estimated average treatment effect of offering participation in formal apprenticeship on earnings is zero. However, we expect that heterogeneity in the employment situation of participants generates some heterogeneity in impacts on earnings. For some youth with limited outside opportunities, participation in formal apprenticeship might lead to an increase in earnings, for example because of the subsidy. For other youth with better opportunities, the impact on earnings might be smaller, and even possibly negative. Figure 3 illustrates impact heterogeneity. We first consider the strong assumption of homogeneous zero treatment effect. The intermediate panel provides the results of the corresponding Mann-Whitney test. The test is implemented using a large number of permutation, which enables to obtain an exact p-value (Imbens and Rubin (2015)). The test has the advantage to being robust to outliers, and clearly rejects homogeneity. The upper panel of the figure displays estimates of the cumulative distributions of potential outcomes in the groups assigned to treatment (blue doted line) and to control (red doted line), as well as the confidence interval of the difference between the two. As can be seen from the figure, the cumulative distribution in the treatment group is first below and then above the cumulative distribution in the control group, meaning that there is no stochastic dominance of one distribution over the other. However, only 71% of youth assigned to the treatment group entered formal apprenticeship. Even if the impact on participation in formal apprenticeship is constant, imperfect take-up can lead

> 32While assignment to treatment has no impact on youth earnings in the short-term, some small positive effects are found on the share of youth able to save. This may be due to the fact that the subsidy is paid regularly into bank accounts. Positive effects are also found on youth self-esteem in the short-term.

26
to the observed patterns in the distribution of potential outcomes. The lower panel of figure 3 presents quantile treatment effects on the population of youth complying with assignment to enter formal apprenticeship.<sup>33</sup> The figure clearly shows a positive and increasing effect for low quantiles, but then a declining pattern and a change in sign. Quantile treatment effects at large quantiles are negative and significant. Although these quantile treatment effects cannot be estimated as effects at quantile (unless assuming rank preservation), the observed patterns are consistent with heterogeneity in treatment effects (see Heckman et al. (1997); Djebbari and Smith (2008)).

# **4.2 Value of Work from Apprentices and Profits in Firms**

We now analyze how the increase in inflow of apprentices indirectly affects firms, including through changes in labor input (time worked), the value of work provided by apprentices, as well as firm profits. During the follow-up survey, firm owners were asked questions about each apprentice who entered the firm since randomization. We measure how these apprentices contribute to firm activities, their hours worked, and whether they are involved in productive tasks. We aggregate apprentice-level measures in each firm across all apprentices who joined since randomization.

Table 5 documents impact on total labor input from apprentices who entered firms since randomization. The total time worked by apprentices at the time of the follow-up survey increase slightly. Firms see a small increase in labor input by apprentices of 6.9 days or 55.9 hours per month. This effect represents a 23 percent increase in days worked by apprentices. These results are significant, but given the size of the increase in the number of apprentices at follow-up (0.613, as can be seen from Table 2), they are actually relatively small. For example, the number of days worked per new apprentice entering firms is 6 _._ 9 _/_ 0 _._ 613 _≈_ 11. One simple explanation for this limited impact is that there are more youth working, but they work less hours. In section 5, we use disaggregated apprentice-level data to show that absenteeism (a pervasive phenomenon for apprenticeship) is in fact higher among formal apprentices.

One critical question pertains to the overall value of work provided by apprentices. In the

> 33We estimate unconditional instrumental variable treatment effects as developed by Frölich and Melly (2013), in which entry into formal apprenticeship is instrumented by the assignment variable.

27
follow-up survey, we ask enterprise owners to recall the work performed by each apprentice during their last working day, and to estimate how much they would have had to pay an occasional worker to accomplish the same tasks. We can then estimate the value of work performed by each apprentice by multiplying this estimated value of work by the number of days worked in the last month. This apprentice-level measure is then aggregated at the firm level across all apprentices who started since randomization. The third column of Table 5 shows that the program led to a strongly positive and significant increase of the value of work by apprentices in treatment firms. The estimated value of work by apprentices increases by 25 543 FCFA per month, a significant 62 percent increase.

Separately, we can estimate the payments made by firms to apprentices (wage bill). The survey describes precisely the types of compensation received by apprentice for meals, transportation, clothing and “motivation”, including both in cash and in-kind payments. We sum all these components at the apprentice level and aggregate again at the firm level. We also compute a net value of work at the firm level by taking the difference between the value of work and wage bill for apprentices. Columns 4 and 5 of Table 5 show that although the number of apprentices substantially and significantly increased, the total wage bill for apprentices did not increase significantly in treatment firms. A small increase in the wage bill is observed as employers provide some payments to program apprentices, but the increase is not significant. As such, the impact on the net value of work (value of work minus wage bill) remains large. This increase amounts to 21 380 FCFA per month, more than doubling the net value of work by apprentices in control firms.

The increase in the net value of work provided by apprentices in firms can be interpreted as marginal profits being positive. It is possible that the net value of work may provide an overestimation of marginal profits, since it accounts for the value of work performed by apprentices and their direct labor costs, but does not take into account indirect training costs incurred by firms. Still, these indirect training costs are unlikely to be large.<sup>34</sup> It thus likely that the marginal value of an additional apprentices is positive, even net of indirect training costs in firms. A positive marginal profit would be consistent with a form of compensation for the training provided by firms to apprentices.

> 34For instance, the number of hours that apprentices spend working under direct supervision of master trainers is rather limited, suggesting youth learn largely by working alongside master craftsmen, inducing limited investments in time spent by master craftsmen solely teaching apprentices.

28
Does the increase in the net value of work by apprentices, or marginal profits, affect average firm profits? Measures of firm sales and profits are notoriously noisy. In the context of the study, we asked firm owners to directly report sales and profits (as recommended by De Mel et al. (2009)). We also asked firm owners to list all their sales from the previous months to obtain a second measure. Based on this listing, we then asked them to report again sales and profits, which provides a third (and our preferred) measure. Moreover, experienced supervisors conducted near-systematic back-check of firm surveys to obtain repeat measures (for 598 out of 677 firms). In total, this means we have six measure _yim, m ∈{_ 1 _, . . . ,_ 6 _}_ of sales and profits for most firms in the sample. We estimate the following regression pooling all the observations together, and including dummies specific to each type of measure: Results are presented in Table A9. As for results on firm workforce, we do not detect any significant effect on average sales or profits, either using the variables themselves or an inverse hyperbolic sine transformation. These results suggest that, while marginal profits increase, no significant impacts are found on average firm profits. This could be due in part to more limited statistical power given the dispersion of the profit variables.

Figure A3 provides additional information on impacts on the distribution of revenues and profits. It shows that the point estimates for program impacts on firm profits and revenues are positive for most of the distribution. However, while these effects are significantly positive in a few parts of the distribution, they are not significant overall. They also tend to be negative at the top of the distribution, driven by a few firms in the control group. The observed patterns are consistent with large variability in the profit and revenue variables. To put results in perspective, the estimated increase in the net value-of-work provided by apprentices (21 380 FCFA per month) amounts to 13 percent of average firm profits in the control group. Overall, the results are suggestive that the increase in the value of work by apprentices (or marginal profits), while strongly significant and positive, are not sufficient to increase average revenues and profits in firms.

29
# **4.3 Summing up Impacts on Earnings**

As in section 3.3, we now re-express ITT estimates as LATE estimates for impacts on earnings per formal apprentice placed by the program. Again, we consider direct impacts on youths and indirect impacts on firms.

We first estimate impacts on youth who were in formal apprenticeships at some point since randomization. We simply estimate parameter _bI_ of the following equation: where _fi_ captures participation in formal apprenticeship (since the start of the experiment), which we again instrument by the youth treatment assignment variable.

On the side of firms, we measure the impact of the entry of one formal apprentice on the “net value of work” in firms as parameter _bS_ of the following equation:

## (7)

where _e_<sup>_f_</sup> _i_<sup>standsforthenumberofformalapprenticesenteringfirms.Again,weusefirm</sup> treatment assignment as an instrument.

However, other indirect effects also need to be considered when accounting for program impacts on youth and firms. As seen in section 3.3, the entry of program apprentices in treatment firms crowds out some traditional apprentices and leaves some other firms with unfilled positions. These flows should also be taken into account in a comprehensive costbenefit analysis. They are, however, complicated to value.<sup>35</sup> Table 6 presents the results. The upper panel presents the reduced-form estimates discussed earlier. The lower panel presents instrumental variable estimates, which are simply the ratio of the first two rows in the upper panel. The first column of the table presents results for youth and the second column results for firms. Results show that, unsurprisingly, there is no significant impact on youth, with a non-significant reduction of FCFA -1 977

> 35Assuming _py_ is the probability of a youth finding an apprenticeship position, and _wa_ and _w_ 0 his/her earnings when finding or not, the contribution of crowded-out youth is _σ_ (1 _− py_ )( _w_ 0 _− wa_ ). Similarly, for firms with unfilled positions, contribution to the net value of work would be _−ω_ (1 _− pv_ )( _ya − wa_ ), where _pv_ is the probability for an apprenticeship position to be filled, and _ya − wa_ the net value of the work of such a position.

30
of total earnings per youth entering formal apprenticeship. On the other hand, at the firm level, the increase in the net value of work is positive, large and significant, with a value of FCFA 27,165 per formal apprentice. The sum of the two effects is FCFA 25 188, which is not significantly different from the subsidy of 30,000 FCFA paid to program apprentices.

Thus, at first glance, even if program impacts on average firm profits are not significant, the estimated net value-of-work provided by apprentices or marginal profits are nearly equal to the subsidy paid by the program. This suggests that indirect effects on firm benefits are nearly sufficient to make the program cost-effective in the short-term. However, there are several considerations to keep in mind. First, as previously explained, there are potential (and likely negative) additional indirect effects on both youths and firms outside the experiment. Recall that for each apprentice hired there is _σ_ = 0 _._ 227 traditional apprentice crowded out and a windfall effect of _ω_ = 0 _._ 259 that we interpret as an increase in unfilled apprenticeship positions. We are unable to value the associated losses, but they are likely to reduce the net surplus documented above. Second, we do not account for the training costs incurred by firms, although we do not expect them to be high, as already mentioned.

# **5 Additional Mechanisms**

We now turn to analyzing a range of additional mechanisms that shed further light on the results. First, as we have shown, the intervention leads youths to enter apprenticeships, which some of them would not have done absent the program. There is a new population of apprentices entering firms, which partly substitutes for a population of traditional apprentices. These populations can be compared to document patterns of selection into (formal) apprenticeships. Second, we analyze the performance of formal apprentices in firms, to better understand their higher productivity and absenteeism. Third, we show how the program (and the wage subsidy) affected the contractual relationship between apprentices and firms.

## **5.1 Youths’ Selection into Apprenticeship**

We first explore the characteristics of youths who entered any form of apprenticeship ( _a_ = 1) thanks to the intervention. Are those youths similar to those who would have entered apprenticeship anyway? This can be analyzed by comparing youths who participate in ap31
prenticeship in the control group (called "Always-takers"), youths in the treatment group who enter apprenticeship due to the intervention (called "Compliers"), and youths in the treatment group who do not enter apprenticeship (called "Never-takers"). We are particularly interested in the comparison of Compliers and Always-takers.<sup>36</sup> Results are presented in Table 7. We use the same baseline characteristics as when checking balance between the treatment and control groups. The first column shows the mean of characteristics for Always-takers, and the second column the estimated mean for Compliers. For completeness, the third column presents the mean for Never-takers. The last column contains the p-value for the test of equality in means between Compliers and Always-takers. The variables are organized by domains: demographics, skills, employment and earnings, aspirations and jobs search, socio-economic background and financial constraints.

The program can affect the decision to enter apprenticeship through two main channels. The first channel relates to expected long-term gains from apprenticeships. There is a distribution of potential gains in the population. In a classical framework, an individual decides to participate if expected returns are larger than a given threshold. The provision of a subsidy should then push some youths with lower expected returns to participate. Assuming expected returns are negatively correlated with employment opportunities, Compliers would be expected to have better opportunities than Always-takers. The table does not provide strong support for this. The only noticeable difference is that the share of Compliers aspiring to become self-employed is lower. It is not clear, however, that this indicates more limited economic opportunities.

A second channel through which the program can affect youths’ decision to enter apprenticeship is by relaxing financial constraints. This is usually one of the main motivations to offer subsidies. In this case, Compliers would be expected to be more financially vulnerable than Always-takers. Results do not provide evidence that compliers face more financial constraints or have lower socio-economic background. However, this is only based on measures of financial constraints from baseline. One of the key findings from the previous section is

> 36Following Abadie (2003), the average characteristic _x_ of "Compliers" is obtained through the regression of _ax_ on _a_ using _T_ as an instrumental variable. The "Always-taker" population is directly observable as those for whom _a_ = 1 and _T_ = 0. Testing the equality of means between Compliers and Always-takers is simply obtained as the test of _αaT_ in the regression _E_ ( _x|a, T_ ) = _αaT at_ + _αT_ (1 _−a_ ) _T_ (1 _−a_ )+ _α_ (1 _−T_ )(1 _−a_ )(1 _−T_ )(1 _−a_ ), in which the excluded category is _a_ (1 _− T_ ) = 1, the "Always-taker" population and _aT_ = 1 identifies the population of Always-takers and Compliers.

32
that entering apprenticeship entails large opportunity costs in terms of forgone employment opportunities. Since there are few financial instruments to smooth consumption over the long duration of an apprenticeship, these opportunity costs are likely the main constraint in access to apprenticeship that the program addresses.

We can also document changes in the population of youths entering firms as apprentices. The follow-up survey asks each apprentice in firms a set of questions about their background characteristics. Three main populations of youths can be compared: formal apprentices _f_ , traditional apprentices in treatment firm _tT_ and traditional apprentices in control firms. We use the following regression to describe heterogeneity in the apprenticeship population: (8) Traditional apprentices entering control firms constitute the excluded category. The two important coefficients in this regression are _bf_ and _btT_ . The first compares the population of formal apprentices with the population of traditional apprentices in control firms, which helps document the selection effect. The second parameter compares the population of traditional apprentices between treatment and control firms. This allows to document the characteristics of youth crowded out of apprenticeships by the entry of formal apprentices.

The top panel of Table 8 presents the results. It shows that formal apprentices are older (by 2.10 years), and more likely to be women (by 11 percentage points) than apprentices in control firms. Formal apprentices also have a higher education level: compared to traditional apprentices in control firms, formal apprentices are 48 percentage points less likely to have no education, 35 percentage points more likely to have completed primary school, and 13 percentage points more likely to have completed lower secondary school. The program therefore places youths that are more educated than the workforce traditionally hired by firms. There is no indication that the program helped insert youths with more limited networks, or more disadvantaged socio-economic backgrounds. They are also no more or less likely to know a master craftsmen, or have a master craftsman as a relative or family acquaintance. Formal apprentices do not report having been more affected by the crisis period in the country either.<sup>37</sup>

> 37The table also shows that there are very few differences in characteristics between traditional apprentices in treatment and control firms (see row labeled "Traditional" in top panel). This suggests that the crowding-

33
# **5.2 Apprentices’ Performance**

Section 4.2 showed that, despite an increase in the total number of apprentices entering firms, there is only a weak increase in the total number of days worked, but a large increase in the value of work by apprentices in firms. In this section, we further explain these results by looking at disaggregated data at the apprentice level. The upper panel of Table 8 showed results estimating equation 8 with dependent variables for apprentice attributes. The purpose was to document selection into apprenticeship. The middle panel of Table 8 presents the results of the estimation of equation 8 on a set variable obtained from the follow-up firm survey. This allows documenting differences by type of apprentices at the moment of the follow-up survey. These differences cannot be given a causal interpretation. They can be explained by selection effects, by some of the effects of program participation, but also by the difference in experience across apprentices.<sup>38</sup> . Still, these results are useful to shed further light on the aggregate impacts observed in firms.

We first analyze apprentices’ participation in firm activities and their productivity. We find two main results. First, absenteeism is more important among formal apprentices than other apprentices: traditional apprentices in control firms worked on average 20.14 days, but the average for formal apprentices is smaller by 7.09 days. Second, formal apprentices are more productive than traditional apprentices in control firms. The value of tasks performed the last day of work is on average FCFA 1,296 for traditional apprentices in control firms. It is higher by FCFA 839 for formal apprentices.<sup>39</sup> To analyze differences in performance, we build several indices of skills: a technical skill index, a behavioral skill index and a learning skill index.<sup>40</sup> Results show that formal out taking place was not associated with stronger selection of a particular profile of apprentice.

> 38The inflow of formal apprentices took place within six month after the start of the experiment, while other apprentices entered over the whole 20 month period (see figure 1).

> 39The table shows that traditional apprentices who entered treatment firms are also more productive than traditional apprentices in control firms. There might be various explanations for the differences in performance. First, employers might have selected traditional apprentices more carefully in treatment firms (particularly since substitution effects are observed within these firms, as we already documented). However, results in the upper panel of Table 8 do not suggest strong selection effects. Very few significant differences are observed between traditional apprentices in treatment and control firms. Second, formal apprentices are found to be more productive, and this may spill over to traditional apprentices.

40 Skills are measured using a set of questions asked to the employer about each apprentice. The technical skills measure includes two general questions about how well apprentices master techniques, tools and safety procedures. It also includes questions specific to each trade: for each trade, we worked with the national training agency (AGEFOP) to identify a list of 2 to 7 technical tasks and asked the employer how well each apprentice performed these tasks (on a scale from 0 to 10). The apprentice-level technical skill index is the average of the scores obtained across the trade-specific questions and the two general questions. The learning 34
apprentices in treatment firms have higher technical skills than traditional apprentices, which can contribute to explain their higher productivity.

The results can also explain explain the higher absenteeism observed among formal apprentices. One striking result is that employers rate formal apprentices as having much lower behavioral skills than traditional apprentices. Looking into the components of the index, the items that drive the results are related to absenteeism and punctuality. There are various possible explanations for a higher absenteeism of formal apprentices. First the program might attract youths who are less interested in apprenticeships in the first place. A second explanation is that the program might not meet youth expectations. The bottom panel of the table shows levels of satisfaction reported by apprentices in treatment and control firms. Formal apprentices are more dissatisfied in general than traditional apprentices, and this is largely driven by a dissatisfaction with the level of earnings received from firms. Interestingly, they are less satisfied with their labor earnings from apprenticeship than with their income in general. These results are consistent with the impacts on earnings documented above.

# **5.3 Contractual Relationships between Employers and Apprentices**

The program affects the bargaining relationship between youths and firms. Indeed, youths receive lower payments from firms than traditional apprentices. We now document how the program modified the contractual arrangements and payments between apprentices and firms. The middle panel of Table 8 details the financial arrangements between employers and youths or their families. Traditional apprenticeship arrangements involve the apprentices (or their family) paying a fee to the master craftsmen. Over time, firms start compensating apprentices, with payments divided between a regular payment (said to be for “soap”) to cover transport costs, room and board, and a “bonus” payment to motivate apprentices. The payment for transport, room and board is often made weekly, and can be partly in kind (e.g. meals). The “bonus” payment is typically paid monthly.

Results show that formal apprentices pay significantly lower fees to firms compared to skills and behavioral skills indices each average several general questions. For the learning skills index, these include: ability to learn, quantity of work, quality of work, speed at work. For the behavioral skills index, these include attitude at work such as: absenteeism, punctuality, respect of clients and boss, seriousness and motivation.

35
traditional apprentices in treatment and control firms. The fact that master craftsmen do not charge a fee is consistent with the program intent in subsidizing access. Employers were requested not to charge fees. In parallel, firms make lower payments to formal apprentices, and these lower payments are mostly driven by a decrease in payments for transport, room and board. The firms internalize that the program subsidy covers such costs, so they strongly reduce their contribution. This behavioral response from firms implies that part of the program subsidy is in fact transmitted to firms. Formal apprentices still receive similar “bonus” payments than traditional apprentices in treatment firms. This shows that firms complement the subsidy offered by the program and attempt to directly motivate apprentices. The decrease in overall payment by firms is consistent with dissatisfaction with labor earnings among apprentices, as reported above.

The table also provides some information on the indirect costs incurred by firms to train apprentices. These costs are key to understand firms’ training decision (Acemoglu and Pischke (1999)). The interpretation of the increase in the net value of work provided by apprentices in firms as marginal profits depends on the magnitude of indirect training costs, as mentioned above. The follow-up survey asked apprentices about the number of hours they spent working independently, under direct supervision of master trainers or watching their master. The lower panel of Table 8 presents the results. In the control group, youth spent on average 2.5, 2.6 and 1.7 hours during their last day of work, working independently, under the supervision of their master, respectively watching him/her. These figures suggest that youth learn mostly by working alongside master craftsmen, inducing limited investments in time spent solely teaching apprentices from master craftsmen. There are also few differences between the various types of apprentices.

# **6 Conclusion**

Evaluations of employment programs usually focus on direct impacts on participants, but these programs can have a range of indirect effects that are rarely taken into account. This paper quantifies the net number of new positions created by a subsidized apprenticeship program in Côte d’Ivoire. We report results from a double-sided randomized control trial specifically designed to measure direct effects among youths and indirect effects in firms.

36
Results show that the apprenticeship program leads to an increase in youth participating in apprenticeship by 52.8 percentage points. This increase accounts for a significant windfall effect: 26% of youth placed in formal apprenticeships actually substituted out of traditional apprenticeships. On the side of firms, the program leads to an increase in the entry of formal apprentices. Substitution effects are also observed, however, as firms hire 0.23 less traditional apprentice per formal apprentice placed. Overall, the net number of apprenticeship positions created by the program is between 51 and 74 percent of individuals placed. We interpret these results as showing that indirect effects are meaningful. Still, the magnitude of the substitution effects is moderate, and it is relatively far from full substitution. Following Calmfors (1994), this suggests that firms are off their labor demand curve or that subsidized and non-subsidized apprentices are imperfect substitutes.

Results point to high opportunity costs as an important constraint for youth to participate in apprenticeships. Youth reorganize their activities to enter formal apprenticeships, and the net average impact on youth earnings is zero in the short-term. Participating in apprenticeship has large opportunity costs in terms of forgone earnings from wage of selfemployment. By providing wage subsidies, the program increases the flow of young people who are able to afford these opportunity costs. This finding highlights another failure of the traditional apprenticeship system that the public program addresses.

Results also show that there is an increase in the number of apprentices entering firms and that marginal profits increase. This suggests that firms are off-equilibrium and constrained in the recruitment of new apprentices, with an excess demand at the wage they offer. A natural question is why firms do not increase the wage they offer to apprentices. Our experiment cannot identify the specific underlying market friction, and this topic would deserve additional research. However, findings are consistent with the mechanisms highlighted by Hardy and McCasland (2015), who show that entry fees can serve as a self-selection device. A low wage might also work as a device to select more motivated youths. Offering the subsidy eliminates this selection mechanism, and potentially implies that the program leads to an inflow of formal apprentices who are less motivated than traditional apprentices. We find consistent evidence that formal apprentices are less assiduous and are less satisfied than traditional apprentices.

An important contribution of the paper is to document how formal apprenticeships have 37
indirect benefits for firms that host apprentices. The net value of work provided by apprentices, which can be interpreted as marginal profits, strongly increases. This is consistent with the framework by Acemoglu and Pischke (1999) showing that firms provide training if they can capture some of its benefits. The increase in marginal profits is of substantial economic magnitude. In fact, this indirect effect may be sufficient to make the program cost-effective in the short-term. Direct impacts on youth earnings are not significant. The finding that a "supply-side" apprenticeship program has impacts on firms on the "demand-side" of the labor-market is important. It shows that indirect effects need to be quantified to provide a robust assessment of program performance. It also shows that human capital interventions targeting youths can have broader benefits for the economy.

38