metadata: authors: 0: author: O'Connell,Stephen A.
1: author: Mation,Lucas Ferreira
2: author: Bevilaqua Teixeira Basto,Joao
3: author: Dutz,Mark Andrew
count: Brazil
docna: 0: docna: Can business input improve the effectiveness
            of worker training? evidence from Brazil's Pronatec-MDIC
docty: Policy Research Working Paper
subtopic: Labor Markets,Educational Sciences,Employment and Unemployment,Rural Labor Markets
teratopic: Social Protections and Labor
abstracts: cdata!: This study evaluates the employment
            effects of a publicly-run national technical vocational
            education training program in Brazil that explicitly takes
            input from firms in determining the location, scale, and
            skill content of courses offered. Using exogenous course
            capacity restrictions, the study finds that those completing
            the course following receipt of a course offer have an 8.6
            percent increase in employment over the year following
            course completion. These effects come from previously
            unemployed trainees who find employment at non-requesting
            firms. The demand-driven program's effects are larger
            and statistically distinguishable from those of a broader
            and institutionally-similar publicly-administered skills
            training program run at the same time that did not take
            input from firms. The study finds that the demand-driven
            program better aligned skill training with future aggregate
            occupational employment growth -- suggesting the input from
            firms captured meaningful information about growth in skill
            demand. Courses offered in occupations that grew more over
            the year following requests exhibited larger employment
            effects, explaining the effectiveness of the demand-driven model.
geo_reg_key: 517191
repnme: repnme: Can business input improve the
            effectiveness of worker training? evidence from
            Brazil's Pronatec-MDIC
display_title: Can business input improve the
            effectiveness of worker training evidence from Brazil's Pronatec-MDIC
listing_relative_url: /research/2017/07/27867384/can-business-input-improve-effectiveness-worker-training-evidence-brazils-pronatec-mdic
topicv3: Education, Skills Development and Labor
            Market m1326135 1484,Labor Markets m1326488 855,Education m1327048 470,Jobs m1503931 2670,Unemployment m1504051 2747
publishtoextweb_dt: 2017-07-31T00:00:00Z
historic_topic: Social Protections and Labor
no_of_pages: 63
author_keys: 0: author_key: 141960
1: author_key: 1443858
2: author_key: 92214
model_extractions: WPS8155 Policy Research Working Paper 8155 Can Business Input Improve the Effectiveness of Worker Training?

Evidence from Brazil’s _Pronatec-MDIC_ _Stephen D. O’Connell Lucas Ferreira Mation João Bevilaqua T. Basto Mark A. Dutz_ Trade and Competitiveness Global Practice Group July 2017
Policy Research Working Paper 8155

# **Abstract**

This study evaluates the employment effects of a publicly-run national technical vocational education training program in Brazil that explicitly takes input from firms in determining the location, scale, and skill content of courses offered. Using exogenous course capacity restrictions, the study finds that those completing the course following receipt of a course offer have an 8.6 percent increase in employment over the year following course completion. These effects come from previously unemployed trainees who find employment at non-requesting firms. The demand-driven program’s effects are larger and statistically distinguishable from those of a broader and institutionally-similar publicly-administered skills training program run at the same time that did not take input from firms. The study finds that the demand-driven program better aligned skill training with future aggregate occupational employment growth—suggesting the input from firms captured meaningful information about growth in skill demand. Courses offered in occupations that grew more over the year following requests exhibited larger employment effects, explaining the effectiveness of the demand-driven model.

This paper is a product of the Trade and Competitiveness Global Practice Group. It is part of a larger effort by the World Bank to provide open access to its research and make a contribution to development policy discussions around the world. Policy Research Working Papers are also posted on the Web at http://econ.worldbank.org. The authors may be contacted at mdutz@worldbank.org.

_The Policy Research Working Paper Series disseminates the findings of work in progress to encourage the exchange of ideas about development issues. An objective of the series is to get the findings out quickly, even if the presentations are less than fully polished. The papers carry the names of the authors and should be cited accordingly. The findings, interpretations, and conclusions expressed in this paper are entirely those of the authors. They do not necessarily represent the views of the International Bank for Reconstruction and Development/World Bank and its affiliated organizations, or those of the Executive Directors of the World Bank or the governments they represent._ Produced by the Research Support Team
# Can Business Input Improve the Effectiveness of Worker Training? Evidence from Brazil’s _Pronatec-MDIC_

Stephen D. O’Connell, Lucas Ferreira Mation, Jo ̃ao Bevilaqua T. Basto, Mark A. Dutz July 31, 2017

**Keywords:** Vocational education and technical training; Training programs; Business development services; Labor demand; Unemployment; Brazil. (JEL J24, J23, J31, J68, J62, M53)

**Author affiliations and contact:** O’Connell: Corresponding author; Department of Economics, Massachusetts Institute of Technology (MIT) and the IZA Institute of Labor Economics, soconnel@mit.edu; Ferreira Mation: Instituto de Pesquisa Econˆomica Aplicada (IPEA), lucas.mation@ipea.gov.br; Bevilaqua: World Bank, jbevilaqua@worldbank.org; Dutz: World Bank, mdutz@worldbank.org.

**Acknowledgments:** The authors are grateful for invaluable research assistance by Nicolas Soares Pinto and Pui Shen Yoong. The authors thank several colleagues for helpful input and discussions, including Onur Altında ̆g, John Giles, Aguinaldo Nogueira Maciente, David McKenzie, Pedro Olinto, Truman Packard, and Romero Rocha, as well as participants in seminars at The World Bank, Pontificia Universidad Javeriana, the 2016 Jobs for Development Conference, Secretaria Especial do Assuntos Estrat ́egicos - Presidˆencia da Rep ́ublica, and the Minist ́erio da Ind ́ustria, Com ́ercio Exterior e Servi ̧cos. Staff at MDIC and MEC were instrumental in providing data and details of program administration. The study was conducted in partnership with the Instituto de Pesquisa Econˆomica Aplicada (IPEA) and funded as part of Brazil’s Productivity Programmatic Approach (P152871) at the World Bank. Views expressed are those of the authors and not of any institution with which they are associated.
# **1 Overview**

A primary objective of active labor market policy is to shape a workforce that matches and continuously upgrades the skill distribution to the evolving labor demand of enterprises. Vocational training programs remain among the most common incarnation of active labor market policies despite a sizable historical literature providing only mixed evidence as to their effectiveness (Card et al. (2015), Kluve (2010), and Heinrich et al. (2013), among others). Betcherman et al. (2004) provide an early synthesis of research of over 150 evaluations of active labor market programs, suggesting a cautious approach in what such policies can “realistically achieve.” Whether the design of publicly-funded skills training can increase its effectiveness is particularly important for policymakers choosing whether to start, adapt, or abandon such programs. In this paper, we investigate the employment returns to a large-scale, publicly administered, technical skills training program in Brazil that explicitly takes input from firms in determining course offerings. To our knowledge, ours is the first study to evaluate a “demand-driven” training program in which firms provide direct input to the government as to the type, volume, and location of skills in demand by private-sector businesses. Because of the unique institutional setting, we then compare program effects to those of an otherwise-similar national skills training program run in parallel within the same institutional and logistical context. This allows us to more meaningfully gauge the potential for a demand-driven design to improve program effectiveness.

Using comprehensive monthly administrative panel data on employment, estimates controlling for individual-level unobservables suggest the program increased the employment rate of trainees by approximately two to three percentage points in the year following the course. To address the endogeneity of course completion, we use exogenous restrictions in course space availability across individuals to implement a difference-in-differences IV 1
strategy estimating the causal effects of training. Some registrants were able to attend and complete the course while others who registered for the course were not allowed to attend due to class oversubscription. The IV estimates suggest those completing the course following receipt of a course offer had an eight-percentage-point higher employment rate than those who did not receive a course offer. This effect is statistically distinguishable from the concurrent technical skills training program without demand-driven features, which had a negligible effect on employment. The demand-driven program is more effective in nearly all subgroups based on course and trainees characteristics. We then explore differences in the composition of courses offered in the two programs, finding that business input improves the matching of course supply to future growth in private sector skill demand.

The current study builds on the literature in a number of ways. First, and most importantly, we evaluate a skills training program that explicitly elicited demands from businesses for the skills and volume of traineeships required in a given area to serve their needs. Because of this, we refer to our focal program, _Pronatec-MDIC_ , as being _demand-driven_ . Second, the institutional setting allows us to compare the program’s effects across types of trainees within _Pronatec-MDIC_ courses, and to similar courses provided outside the demand-driven segment. That is, we investigate differential employment effects for firm-supplied trainees compared to broader sets of registrants concurrently attending the same business-requested courses, including unemployment beneficiaries and those who self-registered. Third, we compare employment effects of the business-requested courses to those from a national training program in which businesses did not provide input into the choice of program course offerings. We provide credible evidence in making these comparisons by using the same empirical strategy that addresses endogenous course completion in both programs. We also eschew common concerns over post-program sample attrition through the use of comprehensive administrative social security data containing monthly employment and earnings information available for all formal sector workers in Brazil.

2
The following section places the current study in the recent literature evaluating similar programs. We then provide further details on the _Pronatec_ and _Pronatec-MDIC_ programs and describe the program records we used to link the course requests, courses held, and student records to the administrative data. We then discuss our empirical strategy and present results. We discuss why the program likely exhibited greater effectiveness in generating employment among trainees, and provide concluding observations on the program and thoughts for future work.

# **2 Background: Technical vocational education and training around the world**

Evidence of the effectiveness of skill training programs remains quite mixed. In a metaanalysis of nearly 100 studies, Card et al. (2010) conclude that skills training programs can effect employment gains in the medium- and long-term, although many programs are ultimately ineffective in reducing unemployment. A host of studies have found widely varying degrees of effectiveness of publicly-provided skill training. Positive employment effects have been found among programs in Peru (N ̃opo et al., 2007; D ́ıaz and Rosas Shady, 2016), Colombia (Attanasio et al., 2011; Kugler et al., 2015), Liberia (Adoho et al., 2014), Nepal (Chakravarty et al., 2015), Malawi (Cho et al., 2013), and Kenya (Honorati, 2015). Other programs have shown negligible impacts, including those in Argentina (Alzu ́a and Brassiolo, 2006), Germany (Caliendo et al., 2011), Dominican Republic Card et al. (2011), Kenya (Hicks et al., 2013), Jordan (Groh et al., 2016), and in an RCT in Turkey (Hirshleifer et al., 2016).<sup>1</sup> Specifically in Brazil, Reis (2015) provides a comprehensive overview of the history of training programs, which began as early as 1942. He analyzes the impact of _PLANFOR_ ,

> 1For earlier reviews, see Betcherman et al. (2004) and Kluve (2010).

3
a worker training program that began in the 1990s, finding short-term effects of the training program of approximately one percentage point increase in employment and seven to eight percent increases in monthly wage rates. Related to the current study, Corseuil et al. (2012) evaluate an apprenticeship-based youth employment program in Brazil, finding that apprentices have a higher probability of getting a formal job in the years after the program.

Causal evaluations of vocational training are available for a newer generation of programs that focus on disadvantaged youth, offer training through private providers, and combine coursework with an internship or work experience at a private firm. Despite remarkably similar in program design, the rigorous experimental evaluations of such programs that exist for Colombia and the Dominican Republic find markedly different results. Attanasio et al. (2011) show that this training in Colombia improved women’s, but not men’s, employment and wages. While their original study was based on a surveyed sample, a more recent study that uses administrative records to track the full applicant pool finds positive effects for both men and women in the short and long run (Attanasio et al., 2017). Results for the Dominican Republic (DR) indicate no effect on employment and only very modest effects on earnings Card et al. (2011). A follow-up study indicates positive results in terms of tested skill acquisition and short term employment for women, a negative effect on employment and no skill acquisition among men, and no long term effects except on women’s expectations and self-esteem (Acevedo et al., 2017).

A strand of the recent literature has argued for the effectiveness of new approaches to reduce unemployment, including cash or capital injections to small businesses or providing business training to potential entrepreneurs (see Blattman and Ralston (2015) for an in-depth review of this literature). Another dimension of this debate focuses on whether existing programs can be reformulated to more closely involve the private sector in determining which skills to provide (World Bank, 2013). Although there are many means by which skills training can be aligned more closely with the needs of the private sector, few of the proposed 4
formulations have been rigorously evaluated in a context where effectiveness across program types can be compared. Our paper contributes to the above studies in several ways. First, the credible studies cited above argue that the work experience component obliges course providers to offer courses for which there is actual demand. However, this feature is a singular design aspect of these programs. Therefore, the importance of the demand-driven design cannot be compared to traditional programs in the same context. In our context, we can compare the effectiveness of a demand-driven and a non-demand driven program, both of which are provided by quasi-public institutions. Second, we evaluate a national program at scale, serving several hundred thousand trainees in the demand-driven segment and more – than one million trainees in the main program segment which are orders of magnitude larger than existing studies. This aspect allows us to maintain statistical power even when disaggregating by course and student characteristics. We also evaluate a program that is not explicitly focused on youth, which, in addition to being a national program, allows us to make broader claims of external validity. In the following section, we describe the institutional context and design of the program that is the focus of this study.

# **3 Brazil’s** **_Pronatec_ and** **_Pronatec-MDIC_**

We study a publicly-provided vocational technical skills training program that was among the first to directly incorporate private sector demand into its design and provision of skills training, and compare its performance with a concurrent, traditionally-designed vocational technical skills training program. In 2011, the federal government of Brazil created the National Program for Access to Technical Education and Employment ( _Pronatec_ ). The program was launched at a time of falling unemployment, with the goal of raising the earnings and employability of lower-income segments of the workforce through participation in the formal labor market. Through late-2015 the program had enrolled more than eight million 5
people in “initial and continuing training courses” that carry a moderate in-class training component (less than 500 hours) and train in skills relevant to a particular occupation. The structure of _Pronatec_ is such that several federal ministries administer and contribute to course and trainee selection.

– The focus of this paper is _Pronatec-MDIC_ the segment of _Pronatec_ administered by Brazil’s Ministry of Industry, Foreign Trade and Services (hereafter MDIC). The unique feature of _Pronatec-MDIC_ is its “demand-driven” nature: MDIC receives explicit requests for specific skills training courses from individual businesses. This program began with only a limited number of training courses in 2013, greatly expanded in 2014, and drastically reduced its scale in 2015 due to federal budget constraints. In 2014, more than 2,000 firms applied for more than 16,000 skills training courses to be given to current or prospective employees across a wide range of industries and occupations. Ultimately more than 300,000 prospective trainees were registered for _Pronatec-MDIC_ -requested courses in 2014 or 2015, with approximately 40 percent of these registrants completing their training course. Below, we discuss the characteristics of the data sources used to analyze effects of the program.

# **3.1 Program administrative records**

Figure 1 summarizes the process from application to course provision in the _Pronatec-MDIC_ program. MDIC receives course requests through a standardized process in which firms indicate the skills, occupation, or course they are looking to have taught, and the number of “seats” (individuals trained) they would like to be provisioned in the given municipality for which they are making the request. These requests have no pecuniary cost for the requesting firms. The requests go through an initial screening by MDIC staff in terms of their viability and appropriateness.<sup>2</sup> Approximately one half of firms’ requests were denied at this stage.

> 2In correspondence with MDIC staff involved in administering _Pronatec-MDIC_ , we were informed that the review process was made to ensure a reasonable volume of seats requested relative to the firm’s scale and

6
Some firms chose to reapply with a different (lower) number of seats requested, while others did not. The process between a course request and the first day of a course can vary, and is discussed in further detail below.

MDIC provided us a comprehensive listing of the course requests in 2014, which contains a measure of the demand for courses/trainees by requesting firms. Each of these requests includes the number of people the requesting organization would like trained (seats), as well as the name of the company or organization that submitted the request, the company’s tax ID, the course requested, the occupation code related to the course, and the municipality in which the course is requested.

Table 1 shows that there were 16,782 records for course applications to MDIC in 2014. Of these, approximately half (8,340) were approved by MDIC. The average number of seats requested in a given course was approximately 38; the average number of seats in approved courses was only slightly higher at 43.6. Not all requestors were firms, however: 17% of course requests were made either by industry or workers’ associations. Among the firm requestors, we are able to match 97% to administrative employment records (described below) based on either the tax ID or the combination of firm name and municipality.

Requests approved by MDIC were then forwarded to the Ministry of Education, which is the body responsible for the overall administration of _Pronatec_ . The Ministry of Education aggregates course demands across ministries, and this aggregation is further screened according to course viability (i.e., having the minimum number of seats to hold a course in a municipality requested across ministries), technical criteria, and budget availability. The Ministry of Education is also responsible for selecting training providers and for setting the criteria for the registration of students. There are a number of providers that offer _Pronatec_ courses, although the majority are offered by Brazil’s _Sistema S_ , which in principle ensures projected needs. If found excessive, course requests would be denied (as opposed to adjusted) to discourage firms from seeking training that significantly exceeded their projected needs.

7
a certain homogeneity in the training provided.<sup>3</sup> The training providers all receive identical reimbursement from the Ministry of Education at a rate of around 10 Reais (approx. 4 USD in 2014) per student-class hour.

The essential difference between the MDIC program and the rest of _Pronatec_ is that the genesis of the requests that MDIC forwards to the Ministry of Education are responsive to direct input from firms. Course requests from other ministries do not come from an explicit application process but rather from varied processes originating within the requesting ministries without formal consultation with prospective employers. This includes taking input from municipal bodies, social assistance centers, and unemployment benefits (UB) programs. Table 2 presents summary statistics on the firm-requested courses. Appendix Figure 1 shows how firm-requested courses were in more skill-intensive occupations (as measured by the median wage rate in 2012 for the occupation corresponding to the course).<sup>4</sup> More than 25% of courses approved by MDIC were not approved by the Ministry of Education. The average course size (conditional on being approved) had 13 seats.<sup>5</sup> That is, based on the average number of seats per firm request several classes would be held to fill a single course demand. The average class ran for 200 course hours, met for approximately eight hours per week, and lasted between and six months in duration.

In Table 3 we compare characteristics of students in firm-requested courses to students in the rest of the _Pronatec_ program. The student file provides rich information on students’ backgrounds, including employment status, gender, age, and the completion status of the student in the class, as well as additional class-level fields including meeting dates and times

> 3 _Sistema S_ is an amalgamation of quasi-governmental organizations in Brazil that administer low-cost or free professional training courses at schools and learning centers throughout the country.

> 4We find that firm-requested classes were concentrated in more specific skill-intensive sectors, including those covering industrial processes and production, infrastructure, and ICT. Course topics relatively underrequested through MDIC were more likely to be in service-oriented fields (education/social development, tourism and hospitality, and environment and health sectors). _χ_<sup>2</sup> tests across MDIC and non-MDIC courses strongly reject equal distributions.

> 5Note that there are more classes offered than requested, as requests were typically 2-3 times larger than class sizes available.

8
and the institution providing the training. Compared to students in non-MDIC classes, students in MDIC courses are slightly older, more likely to be male, and more likely to have been unemployed at the time of registration, and exhibit higher course completion rates.

Because all records contain unique national identification numbers, we are able to link the students in _Pronatec-MDIC_ courses with national administrative data from the Ministry of Labor containing information on all formal sector employment. In the following section we describe the employment data and how we linked them to trainees in firm-requested courses.

# **3.2 Monthly employment records**

Our employment data come from the Rela ̧cao Anual de Informa ̧c ̃oes Sociais (hereafter RAIS). RAIS is an annual administrative dataset containing employment and earnings information collected primarily for the purpose of administering social welfare programs such as unemployment and retirement benefits. Our data contain full details on the monthly employment and wage earnings of all formally employed workers in Brazil from calendar years 2013 to 2015. RAIS contains employer-reported records of a worker’s hours, earnings, hiring/dismissal dates and reasons (if applicable), as well as the firm’s industrial classification, the worker’s occupational classification, and the worker’s sex, education level, and age. The data importantly contain unique identifiers (i.e., tax IDs) for both workers and employing firms, which are the standard identification provided to and used by firms and workers for their various dimensions of interaction with the government. These fields are used to link the student information to their employment records as well as to identify employment at requesting firms versus other businesses. We deflate earnings using a standard monthly consumer price index (IBGE, 2016).<sup>6</sup>

> 6For workers who have multiple records within a given month, or who worked only part of the month (based on precise hiring and firing dates), we add all deflated earnings across jobs, and construct a monthly wage rate based on the share of the month worked.

9
Table 4 contains summary statistics on the worker-level panel dataset used for analysis. There are 335,300 unique individuals in the administrative data that registered for any of the firm-requested courses, including registrants supplied by requesting firms, unemployment beneficiaries, and others. 60 percent of the sample is male, and 40 percent completed the training course for which they registered. Although only eight percent of the sample was denied a seat for exogenous reasons (which we term “administrative constraints”), 60 percent of all students were in a class that had at least one registrant not complete the course due to these reasons. Nine percent of the students were registered through MDIC by requesting firms, and while the average employment rate was 60 percent, only four percent were employed in requesting firms. The average employment rate across person-months was 57 percent, and the mean real monthly earnings (excluding months unemployed) was approximately 1,250 (in 2012 Reais).

Since courses are typically not exclusive to a particular type of registrant, the firmrequested courses are comprised of a mix of trainees who registered through different channels. Appendix Tables 1, 2, and 3 contain summary statistics on the three focal subsamples of registrants in the firm-requested courses: those who were pre-registered by requesting firms, unemployment beneficiaries, and those who were registered through neither channel.<sup>7</sup> As we expected, and these tables show, these groups of registrants have very different labor market situations: the firm-supplied registrants have higher employment rates (75%, compared to 60% for UB registrants and 53% for others), were far more likely to be employed in requesting firms (23.7% vs. _<_ 2.5% in both other groups), and earned higher wages (R$1,492/month vs. R$1,135/R$1,129). In the analyses below, we estimate effects both for all registrants in firm-requested courses as well for as these three focal subsamples.

> 7 Unemployment beneficiaries are required to register for a _Pronatec_ training course as a condition to receive unemployment benefits in certain cases (for those seeking unemployment benefits for the second time within 10 years).

10
# **4 Empirical Strategy**

Firm-requested courses were offered throughout all months in 2014 and 2015. We exploit differential course timings and the detailed information on the reasons why students did or did not complete the course for which they registered to estimate effects of course completion.

We first estimate course effects on employment in a standard difference-in-differences approach. One concern with this approach is the construction of the counterfactual group, as it is well-documented that the timing of the start of skills training is related to time-variant un– observables in particular, recent unemployment spells (i.e., the “Ashenfelter Dip”). In our context, however, the counterfactual group is constructed of individuals who registered for training courses with specific start dates, but did not attend or complete them. This allows us to estimate a difference in the relative employment across these two groups controlling for common employment patterns relative to the start of the training course.

Because course attendance may have different effects on trainees during the course versus after its conclusion, we segment the analysis window into three time periods: pre-course (before the course begins), during the course, and post-course. The difference-in-differences estimator is then: _Yict_ = _β_ 0 + _β_ 1 _∗ courseict_ + _β_ 2 _∗ postcourseict_ + _β_ 3 _∗ courseict ∗ Completeri_ In equation 1, _i_ indexes individuals registered for class _c_ whose employment is being observed in month _t_ . _β_ 1 and _β_ 2 capture aggregate level differences in employment in the course and post-course period (relative to the pre-course period), _β_ 3 captures the “during course” effect of course completion on employment, and _β_ 4 gives the focal difference-in-differences estimator of the course completion on the outcome. The vector of individual fixed effects in _λi_ absorbs 11
individual-level unobservables (as well as location and classroom effects, and the “main effect” of completion) and _γt_ controls for common (monthly) shocks to the labor market. We then correct for within-class correlations in the error term (i.e., across students taking the same course in the same place for the same period; _>_ 15,000 classes/clusters) and for potential aggregate correlations by month _t_ (36 clusters). We limit the sample to 18 months before and after the start of the course, and estimate equation 1 via OLS.

As a complement to the above approach, we then estimate program effects in an event study framework to better gauge the evolution of effects over varying horizons relative to training. We choose this to accommodate the amount of time it may take for effects to materialize from vocational/technical training (Card et al., 2015), as it allows for an unrestricted examination of the differences in labor market outcomes in the months before and after a worker begins the training course. From the event study analysis, we are then better able to characterize the likely pattern of effects we expect to see during a slightly longer time horizon.

In the event study specification, each month relative to the start of the course is given its own differential effect for completers and is estimated by the following equation: In equation 2, the vector _β_ measures differences in outcomes for completers relative to noncompleters at each period relative to course start, capturing the effect of program completion. Similar to the above, individual fixed effects in _λ_ and month fixed effects in _γ_ control for individual selection on time-invariant unobservables and common aggregate shocks. In the estimation, the course start month (indexed to zero) is the omitted group.

12
# **5 Results**

## **5.1 Completers vs non-completers with individual fixed effects**

We first discuss the main difference-in-differences results, with coefficients from the estimation of equation 1 on the firm-requested-course registrants presented in Table 5. We find a 0.023 percentage-point (or 0.046 standard deviation) increase in the probability of employment for course completers, on average, in the year following course completion. This effect can then be decomposed, across panels B and C, into effects emanating from employment at requesting firms versus employment outside requesting firms, respectively. We find that a small fraction of this effect comes from employment at requesting firms, while the majority comes from higher employment probability at non-requesting firms. In column 2, we condition the estimation just on the sample of firm-supplied registrants, and find that their overall employment effect (0.031) comes largely through increased employment at requesting firms (0.020).<sup>8</sup> We estimate sizable negative effects of course completion on employment among UB registrants (column 3), and small but significant positive effects among all other registrants who find employment at requesting firms (Panel B, column 4).

Event study estimates from the estimation of equation 2 are presented in Appendix Tables 4 to 5, with results plotted graphically in Figures 2 to 10, respectively. In the graphs, series for completers and non-completers correspond to equation 2 vectors _α_ and ( _α_ + _β_ ), respectively. The event study analysis provides some immediately apparent patterns regarding employment and participation in worker training in this context that would otherwise not be apparent from the main analysis. Figure 2 shows how the employment rate of program participants in the month in which they start their course is, for the overall sample, at its – lowest point in the preceding months, confirming that a substantial share more than 40%

> 8It is important to note that many of the registrants already worked for requesting firms prior to the training course, so this estimate is necessarily net of pre-course employment at these firms.

13
– of individuals selecting into the program experienced a job loss in the year prior to the start of their course (commonly known as “Ashenfelter’s dip”). We then see non-completers outpace course completers in their employment rate for the months following the start of their course. There may be a number of reasons for this, the most likely being that job offers received prior to or in the early stages of the course period induce dropout. Nevertheless, the event study provides visual evidence (or not) of the parallel trends assumption in the estimator.

Across the three groups, the employment patterns in the pre-course period are markedly different. Among MDIC-supplied registrants, we find no stark pre-course dip in employment rates (Figure 4). This is likely attributable to the selection of individuals who are registered through the MDIC process who have higher pre-course employment rates than the average trainee and have not been registered for a training course due to recent unemployment. Figure 4 highlights two additionally important points. First, completers are significantly different from non-completers in pre-course employment levels. Second, while employment rates for completers are approximately maintained in the post-course period, those for noncompleters exhibit a flat trend after the course, while completers continue trending upward. The gap in employment rates leads to a nearly ten-percentage-point difference for the year following the end of the course. Among unemployment beneficiaries who also attended firmrequested courses, we find a sharp pre-course reduction in employment rates (Figure 5), reflecting precisely the institutional situation described above. The approximately equal differentials across completers and non-completers before and after the course validate the DD estimates showing that the program had minimal employment effect on these registrants. Table 4 contains a subset of focal coefficients in the _β_ vector capturing the difference in the outcome for the completers relative to non-completers in months before the training begins, in the month of completion, and three, six, nine, and 12 months after the end of the course. Employment effects are statistically distinguishable by the end of the course period (for the overall sample, and MDIC registrants) or within a few months after the completion of 14
the course (for the “all other” registrants), although these point estimates do not account for pre-course differences. This analysis additionally allows us to gauge the likely trend in program effects over our study horizon. Effects appear to be steadily increasing over time (for all but UB registrants), suggesting that estimates in the current analysis may be smaller than those that use a longer horizon.

# **5.2 Identification challenges**

The identifying assumption in the analysis above is that completers and non-completers experienced common shocks in the months prior to the start of the course and would have exhibited similar employment patterns in the absence of treatment. The approach assumes that the endogeneity between course completion and later employment is entirely due to time-invariant individual-level unobservables. This is a strong assumption, precisely because employment gained during the course period might affect whether an individual completes the course. That is, some individuals may receive acceptable employment offers during the course period that cause them to drop the course. Since these offers are not observable, increase employment, and are negatively correlated with course completion, we can then reason that this approach above likely gives a downward-biased estimate of the effect of training on employment. In the next section, we address this concern by using exogenously determined variation in space availability in courses to identify the causal effect of training on employment.

# **5.3 IV estimates using course offer as instrument**

Since we remain concerned about the endogeneity of course completion, we make use of the detailed information on the reasons why students did not complete the courses they registered for to identify students who were kept from attending a course due to exogenous 15
reasons. In general, the reasons for which an individual would register for a course but not be able to complete it could be either personal (e.g., quits and no-shows) or what we refer to as “administrative reasons” – those outside the control of the individual registrant. These typically include a lack of seats due to class oversubscription or space limitations.<sup>9</sup> In the case of course oversubscription, which is the cause of the vast majority of course offer restrictions, the segment of unemployment beneficiary registrants was used to make space for various other groups registered through other channels. This was done on a first-come, first-served basis, although the administrative program data did not retain sufficient detail to observe the precise cutoff for course offer receipt. We use this information on administrative restrictions – to create an instrument for whether a student received a “course offer” taking the value of one for those who had registered for a class and were not restricted administratively from attending the class. Because the analysis is split into three periods relative to course start and end, we then interact the instrument separately with indicators for the during-course and post-course periods. The structural equation remains as in equation 1, where the two endogenous variables, _courseict ∗Completeri_ and _postcourseict ∗Completeri_ are instrumented with _courseict ∗ Offeri_ and _postcourseict ∗ Offeri_ . (Note that the main effects of _Offeri_ are absorbed in the individual fixed effects.)

# **5.4 Test of parallel trends**

One of the central assumptions underlying this approach is whether the two groups satisfy the – “parallel trends” assumption that is, that trends across offer recipients and non-recipients were parallel in the pre-course period and would have continued to be so in the absence of the course. We test the first part of this assumption (whether there were parallel trends prior to the course) by limiting the sample to all pre-course observations and estimating a

> 9We confirmed with the Ministry of Education that the record codes used to identify administrative non-completers always corresponded to reasons for non-completion that were outside the control of trainees themselves; _e.g._ , _force majeure_ cancellations or seat reductions, class oversubscription, etc.

16
slope coefficient on the number of months relative to the course and a slope differential for offer recipients, via: _Yict_ = _α_ 0 + _α_ 1 _∗ months relative to courseict_ In equation 3, _α_ 2 gives the slope differential for offer recipients’ pre-course employment trends. In this case, a positive coefficient would raise concerns about upward bias in the resulting estimates (and vice versa for a negative coefficient). The coefficients from this estimation across the pooled and separated samples are in Table 6. In column 1, the estimated slope differential is small and statistically insignificant at conventional levels. Because of – the precision afforded by the sample, we can reject slope coefficients as small as 0.001 a magnitude still not large enough to present meaningful concern for our estimates below. Even among subsamples there is no statistically significant differential trend among offer recipients, lending support for the parallel trends assumption as applied to the pre-course period.

# **5.5 IV estimates: First- and second stages**

First stage coefficients from the estimation of the focal endogenous variable, _postcourseict ∗ Completeri_ , are in Table 7. The coefficient effectively reflects the completion rate among those offered a seat in the course ( _i.e._ , compliance), which, as expected, is slightly higher than the overall course completion rate. Second stage coefficient estimates are in Table 8. We find average employment effect (Panel A, column 1) at 8.6 percentage points to be substantially larger than the naive difference-in-difference estimates. This implies the reduced form intent17
to-treat effect of course offer receipt is just under four percentage points. We find that the UB registrants (column 3), to whom the instrument was particularly relevant and from whom the IV estimates are largely identified, exhibit large (14.6 percentage-point) employment effects of course completion induced from receiving a course offer. Firm-supplied registrants (column 2) do not appear to exhibit employment effects from this variation, and all other registrants (column 4) exhibit small but significant increases in employment. The majority of the large effects for UB registants (Panel A) are attributable to employment at nonrequesting firms (Panel C). The strong effects among the unemployed finding employment in non-requesting firms may be partly explained by the fact that “lead” firms would submit requests on behalf of smaller, local suppliers – providing further evidence that the courses requested were indicative of general skill shortages experienced by several firms.

We find these effects to be relatively large, especially in view of the mixed results found in previous literature. However, the identifying variation comes from students in courses that had sufficient demand to be oversubscribed; if student demand is at all an indicator of the quality of courses or the employment prospects they generate, we can reason that these estimates will be larger than those for the entire course distribution. Among the types of students taking courses in our context, the UB registrants have the lowest pre-course employment rates and are thus more likely to exhibit effects on the extensive margin of employment compared to, for example, firm-supplied registrants of whom many are employed at the start of the course.

# **5.6 Robustness: Common support sample**

We address further concerns of unobservable differences between offer recipients and those not receiving a course offer by constructing a within-class, common support matched sample. In this process, we take the set of registrants who do not receive a course offer and match them 18
with their nearest neighbor (without replacement) based on offer propensity predicted by a discrete choice model using individual characteristics and detailed employment histories to estimate the offer indicator. This effectively balances the sample to have 50% offer recipients and leaves a final sample comprised of individuals from the set of classes that experienced any capacity constraints. The results from this matched analysis (available up on request) mirror closely the magnitudes for the UB sample in Table 8 of approximately a 15 percentage-point effect on employment.

# **5.7 Estimation of effects from the broader** **_Pronatec_ program**

A major benefit of the Brazilian context is our ability to perform the exact same analysis on an otherwise-similar national skills training program that did not take input from firms in allocating skills training. This program, which was implemented by the same ministries and course providers, existed at the same time as our focal program and essentially ran in parallel to the demand-driven program. Over the study period, the main _Pronatec_ program was approximately six times larger than the demand-driven segment, serving over 1.6 million trainees for whom we can similarly observe complete employment and wage histories in the administrative data. We undertake the same analysis as above on all _Pronatec_ trainees outside the firm-requested courses who registered for a class scheduled to be held in 2014 or 2015 (corresponding to the same period during which the business-requested courses were held).

We estimate the IV specification for individuals who registered for a training course outside of the demand-driven segment. The estimates for the broader _Pronatec_ program are presented in Table 9, and show a markedly different pattern from those of the demand-driven segment: IV estimates of the program effect are negligible overall, and precisely estimated. This masks substantial heterogeneity across UB recipients and the majority of other reg19
istrants: UB registrants saw average increases of eight percent in employment probability following the course (compared to approximately 15 percent in the MDIC program, Table 8, Panel A, column 3), while other registrants saw a nearly ten percent lower employment rate – combining into an overall minimal and statistically insignificant effect on employment.

# **5.8 Effect heterogeneity by skill group, region, and trainees’ formal education**

To better understand effect heterogeneity within and across the programs, we estimate the IV specification across subgroups in four dimensions. In Table 10 we present the IV coefficients for both program segments estimated separately by each of seven major occupational subgroups, five skill levels (based on the 2013 median wage in the occupation trained), five regions of the country, and trainee education level.

We find that even within occupational groups, the demand-driven segment exhibited either similar or meaningfully larger effects in courses in the four occupational groups (Industrial Processes and Production; Management and Business; Information and Communication; and Environment, Health and Education) that comprise approximately three-quarters of individuals trained in either the demand-driven program or _Pronatec_ overall. We then segment courses into quintiles based on the median wage of the occupation for which they were training. The demand-driven program had substantially larger effects than the main program in the lowest-skill courses, as well as middle- and upper-skill trainings (the latter of which exhibited negative effects in the main program). Again, the overall effect is not driven by a compositional effect but by differential effectiveness within skill levels. Finally, the demand-driven program was more effective in four regions of the country (all except the North, where performance was similar). The demand-driven program has a slightly larger share of trainings in the Northeast, where it was almost three times as effective as the main 20
program, compared to a larger proportion of trainings in the Southeast by the main program, where employment effects were negative. By trainee skill level, we find that the demand– driven program was more effective at all levels and was particularly more effective among primary- and middle-school educated trainees.

# **5.9 Can effectiveness be explained by selection on students?**

A final concern is that the differential effectiveness of the two programs may be explained by selection of UB trainees who join the firm-requested courses versus the overall program. We believe this is unlikely for institutional reasons (firm-requested courses were never outwardly advertised or known to applicants or providers to have their provenance from this channel). We show below empirically that this was not the case.

To test for differential selection of UB trainees into the firm-requested courses versus the main program, we take the pooled sample of UB registrants in both programs and estimate an indicator for registering for a firm-requested course in a multivariate regression containing the registrants’ pre-course employment history at different periods relative to the course start. Despite the empirical strategy fully accounting for individual-level unobservables, this test allows us to speak to whether the UB registrants in either program were systematically different from each other in their pre-course employability. We estimate the indicator for being in a demand-driven course with individual employment between one and six months prior to the course start. Results from this estimation are found in Appendix Table 6, and show that, if anything, UB registrants in the demand-driven segment had _lower_ employment rates prior to the course than those in the main program segment.

21
# **5.10 Alternative cross-sectional IV**

Finally, we present below an alternative IV strategy used to validate the main results. Since the assignment mechanism is ultimately cross-sectional, we estimate employment rates at each month after the start of the individual’s course, starting from the month of course completion and up to 18 months after the start of the course. In this approach, we are also able to explicitly control for differential pre-course employment and education levels as well as saturate the specification with pre-course employment indicators.

As before, the first stage estimation reveals the rate at which those with a course offer complete the course, conditional on the included fixed effects and covariates. The crosssectional first-stage equation is given by: where _i_ indexes individuals across courses _c_ . Table 11 presents estimates from the first stage equation. Column 1 contains the first stage coefficient including only a vector of month fixed effects. Column 2 presents estimates of the first stage relationship with a full set of class fixed effects, and Column 3 adds the education level of the individual registrant. We find that the first stage coefficient is stable around .40 across these columns, indicating the course completion rate of approximately 40 percent as shown in Table 4. The first-stage coefficient remains unchanged with the inclusion of a larger vector of class fixed effects, the registrant’s education level, as well as pre-period employment indicators.

The second stage equation is then given by: Table 12 presents estimates of the second-stage coefficient on course completion estimated 22
across varying specifications and months relative to the start of the registrant’s course. As above, we estimate separate specifications including month fixed effects, class fixed effects, and including education and pre-period employment rates as additional controls. The final specification, with a full set of controls for pre-course employment at one, three six, and nine months prior to the start of the course, has slightly larger estimates than the main IV – estimates above although they are sharply decreasing over the study period. Nonetheless, the employment effects are still sizable and, on average, of a comparable magnitude to the main estimates above, and we confirm that these magnitudes are largely driven by the UB registrant sample (Table 13).

# **6 Explaining program effectiveness: Matching skill training to growth in skill demand**

Why did the demand-driven program exhibit such substantially larger employment effects than the larger, non-demand-driven segment? Our primary hypothesis is that the input from firms contained meaningful information that allowed the provision of skill trainings to be better aligned with aggregate growth in skill demand. To assess whether this was the case, we compare the occupational and geographic distribution of trainings in the demanddriven and main program to two “reference” distributions: the cross-sectional distribution of aggregate employment, and growth in employment over the 2013 - 2015 period. We compute the sum of squared deviations (SSD) of the focal training program to the two reference distributions to understand which training program was better matched to the cross-section of employment and which was better matched to aggregate employment growth.

In Table 14 we present the values of the SSD for the main and demand-driven programs compared to the focal reference distributions for three categorical variables: three-digit occu23
pation, state, and state-by-occupation. (Note that higher values indicate greater mismatch.) The conclusions from this analysis are clear. Comparing Columns 1 and 2, the main program was better aligned with the cross-section of employment both geographically and occupationally. Comparing Columns 3 and 4, the demand-driven program was better matched to the _growth_ in skill demand by occupation. (Note also that the programs are, in general, better matched to the cross-section of employment than its growth, given by the lower magnitudes in Columns 1-2 compared to Columns 3-4). We note that this difference between the two programs is particularly pronounced when we focus our reference distribution just on the period of end-2014 to end-2015 (not presented), which comprises a period that occurred following the timing of requests made by firms (rather than spanning the period of – requests) suggesting the requests indicated expected _future_ demand rather than recent trends. This provides further evidence that business input into training programs can be useful in identifying economy-wide skill shortages and future growth in skill demand.

Whether better matching of courses to future employment growth explains differential effectiveness across programs hinges on the existence of differential effectiveness of courses offered in faster-growing occupations. To test whether this is indeed the case, we estimate heterogeneous effects of the program by interacting the instrument and focal endogenous variable with the 2013 to 2015 share of aggregate employment growth corresponding to the occupation of the course attended. We transform this measure to have mean zero and unit standard deviation, and estimates for both program segments are found in Table 15. We find stronger employment effects are among faster-growing occupations in both segments. That is, courses offered in fast-growing occupations exhibited larger employment effects, which explains the effectiveness of the demand-driven model. We also find this is particularly relevant for employment among the vast majority of self-supplied registrants, and that differential effect is larger among the non-demand-driven segment – possibly due to diminishing margins in the degree of program alignment to growth in skill demand.

24
# **7 Conclusion**

The fast-evolving skill demand of employers in both developed and developing countries has the potential to leave those less skilled, less connected, and already economically marginalized even further behind. Occupational training programs are a common active labor market policy with the potential to address skill shortages and mismatch, although their effectiveness across contexts varies greatly. In this paper, we evaluate the effects of a recent large-scale technical skill training program in Brazil that took informational input from firms into consideration for the provision of publicly administered vocational technical training. Our empirical strategy and use of comprehensive monthly labor market data allow us to avoid several of the common challenges in the evaluation of occupational training programs. We find that the program has a substantial causal effect on employment appearing shortly after course completion and persisting for at least one year following the course. Program effects are substantially larger than those in a similarly-run occupational training program that does not take input from firms in deciding the content and scale of skills training. The informational input from firms allows the allocation of skill trainings to better match _future_ growth in skill demand rather than the static distribution of employment, which we show likely explains increased program effectiveness.

There may be an important efficiency-equity tradeoff in demand-driven programs. This is particularly germane in contexts where economic inequality is sought to be attenuated through active labor market policies. Such designs have the potential to exclude segments of firms and workers from accessing program benefits. In particular, we find that the distribution of courses provided under the demand-driven program is concentrated in more skill-intensive occupations that are more likely to find employment in industries offering relatively higher-wage formal employment. Participating businesses may also comprise a selected segment of the distribution of firms. For workers, if initial skill levels play a meaningful role in access or ability to complete these courses, this dimension may serve to exacerbate 25
inequality between program trainees and those not able to access the program’s benefits. Similar effects may occur if firms’ access to the demand-driven program is limited by their size or the scale of their requests.

Nevertheless, our results confirm the potential for partnership between the public and private sectors in investing in skills. We find strong evidence of employment effects among registrants not previously connected to requesting firms who gain employment at non-requesting firms. This suggests two things: the firms requesting skills training are indicating local skills shortages applicable to more than just their own firm, and the trainings requested under the program are likely providing general rather than firm-specific skills. When this is the case, – trainees (as opposed to firms) are the residual claimants to the skills provided suggesting the training program is effective in garnering information about broader local skills shortages and that the ultimate beneficiaries of the training are trainees, rather than partnering firms. This allows for a cautious optimism regarding the potential to expand demand-driven training programs in other contexts in ways that can have pro-poor impacts. Future work in this and other contexts should investigate how demand-driven training programs affect requesting businesses’ labor demand and output expansion, as well as affect workers’ productivity and occupational mobility within and across firms.

26
# **References**

- Acevedo, P., G. Cruces, P. Gertler, and S. Martinez (2017): “Living Up to Expectations: How Job Training Made Women Better Off and Men Worse Off,” _NBER Working Paper No. 23264_ .

- Adoho, F., S. Chakravarty, K. Jr, D. T., M. Lundberg, and A. Tasneem (2014): “The Impact of an Adolescent Girls Employment Program: The EPAG Project in Liberia,” _World Bank Policy Research Working Paper, no. 6832_ .

- Alzu ́a, M. L. and P. Brassiolo (2006): “The impact of training policies in Argentina: an evaluation of Proyecto Joven,” _Inter-American Development Bank Working Paper_ .

- Attanasio, O., A. Guar ́ın, C. Medina, and C. Meghir (2017): “Vocational Training for Disadvantaged Youth in Colombia: A Long-Term Follow-Up,” _American Economic Journal: Applied Economics_ , 9, 131–143.

- Attanasio, O., A. Kugler, and C. Meghir (2011): “Subsidizing Vocational Training for Disadvantaged Youth in Developing Countries: Evidence from a Randomized Trial,” _American Economic Journal: Applied Economics_ , 3, 188–220.

- Betcherman, G., K. Olivas, and A. Dar (2004): “Impacts of Active Labor Market Programs: New Evidence from Evaluations with Particular Attention to Developing and Transition Countries,” _Social Protection Discussion Paper, World Bank, Washington, DC._

- Blattman, C. and L. Ralston (2015): “Generating employment in poor and fragile states: Evidence from labor market and entrepreneurship programs,” Columbia University, Mimeo.

- Caliendo, M., S. K ̈unn, and R. Schmidl (2011): “Fighting Youth Unemployment: The Effects of Active Labor Market Policies. IZA DP No. 6222,” _Institute for the Study of Labor_ .

27
- Card, D., P. Ibarrar ́an, F. Regalia, D. Rosas-Shady, and Y. Soares (2011): “The Labor Market Impacts of Youth Training in the Dominican Republic,” _Journal of Labor Economics_ , 29, 267–300.

- Card, D., K. J., and A. Weber (2015): “What Works? A Meta Analysis of Recent Active Labor Market Program Evaluations,” _NBER Working Paper No. 21431_ .

- Card, D., J. Kluve, and A. Weber (2010): “Active labour market policy evaluations: A metaanalysis,” _The Economic Journal_ , 120, 452–477.

- Chakravarty, S., M. Lundberg, P. N. Danchev, and J. Zenker (2015): “The role of training programs for youth employment in Nepal: impact evaluation report on the employment fund.” The World Bank, Washington, DC.

- Cho, Y., D. Kalomba, A. M. Mobarak, and V. Orozco (2013): “Gender Differences in the Effects of Vocational Training: Constraints on Women and Drop-Out Behavior,” _IZA DP No. 7408_ .

- Corseuil, C. H., M. Foguel, G. Gonzaga, and E. P. Ribeiro (2012): “The Effect of an Apprenticeship Program on Labor Market Outcomes of Youth in Brazil,” Presented in the 7th IZA/World Bank Conference: Employment and Development, New Delhi.

- D ́ıaz, J. J. and D. Rosas Shady (2016): “Impact Evaluation of the Job Youth Training Program Projoven,” _Inter-American Development Bank Working Paper_ .

- Groh, M., N. Krishnan, D. McKenzie, and T. Vishwanath (2016): “The Impact of Soft Skills Training on Female Youth Employment: Evidence from a Randomized Experiment in Jordan,” _IZA Journal of Labor and Development_ , 5, 9.

- Heinrich, C. J., P. R. Mueser, K. R. Troske, K.-S. Jeon, and D. C. Kahvecioglu (2013): “Do Public Employment and Training Programs Work?” _IZA Journal of Labor Economics_ , 2, 1–23.

28
- Hicks, J. H., M. Kremer, I. Mbiti, and E. Miguel (2013): “Vocational education in Kenya: Evidence from a randomized evaluation among youth,” Mimeo.

- Hirshleifer, S., D. McKenzie, R. Almeida, and C. Ridao-Cano (2016): “The Impact of Vocational Training for the Unemployed: Experimental Evidence from Turkey,” _Economic Journal_ , 126, 2115–2146.

- Honorati, M. (2015): “The Impact of Private Sector Internship and Training on Urban Youth in Kenya,” _World Bank Policy Research Working Paper No. 7404_ .

- “<sup> ́</sup>

- IBGE (2016): Indice Nacional de Pre ̧cos ao Consumidor Amplo (IPCA),” Instituto Brasileiro de Geografia e Estat ́ıstica, Sistema Nacional de<sup>`</sup> Indices de Pre ̧cos ao Consumidor (IBGE/SNIPC).

- Kluve, J. (2010): “The Effectiveness of European Active Labor Market Programs,” _Labour Economics_ , 17, 904–918.

- Kugler, A., M. Kugler, J. Saavedra, and L. O. H. Prada (2015): “Long-term direct and spillover effects of job training: Experimental evidence from Colombia (No. w21607),” _National Bureau of Economic Research Working Paper Series_ .

- N ̃opo, H., M. Robles, and J. Saavedra (2007): “Occupational Training to Reduce Gender Segregation: The Impacts of ProJoven,” _Inter-American Development Bank Working Paper_ .

- Reis, M. (2015): “Vocational Training and Labor Market Outcomes in Brazil,” _B.E. Journal of Economic Analysis and Policy_ , 15, 377–405.

- World Bank (2013): “World Development Report 2013: Jobs,” _World Bank Group. Washington, DC._

29
Table 1: Summary Statistics, MDIC course demand applications

|**Variable**|**Mean**|**Std. Dev.**|**Obs**|
|---|---|---|---|
|Whether course was approved [0/1]|0.50|0.50|16,782|
|Number of seats requested|37.8|178.1|16,782|
|Number of seats requested — approved|43.6|82.3|8,340|
|Requestor is worker/industry association [0/1]|0.17|0.37|16,782|
|Whether requestor found in RAIS — frm|0.97|0.17|13,969|

Source: Authors’ calculations using MDIC course demand data.

Table 2: Summary Statistics, MDIC-requested courses

|**Variable**|**Mean**|**Std. Dev.**|**Obs**|
|---|---|---|---|
|Whether course was approved [0/1]|0.74|0.44|35,834|
|Number of seats _|_ approved|16.0|36.5|22,198|
|Course hours _|_ approved|198.4|44.6|26,666|

Source: Authors’ calculations using MEC course data. Note: Sample comprised of records matching course-municipality of requests to MDIC. Less than 1% of approved requests were among courses with greater than 150 seats.

30
||**N**|4,235,525|4,219,858|4,235,525<br>4,235,525|atching course and|
|---|---|---|---|---|---|
|dents|**p-value**|_<_0.001|_<_0.001|_<_0.001<br>_<_0.001|records m|
|stu|**_t_**|.1|.7|.2<br>.2|d of|
|urse||111|108|223<br>156|prise|
|-co|**e.**|01|12|01<br>00|om|
|MDIC|**s.**|0.0|0.0|0.0<br>0.0|ents c|
|non-|**ence**|0.077|1.644|0.160<br>0.090|stud|
|ents vs.|**difer**||||C course|
|tud|**IC**|322|629|384<br>182|DI|
|e s|**D**|0.|7.|0.<br>0.|M|
|DIC-cours|**C**<br>**non-M**|99|72<br>2|44<br>73|ata. Note:|
|: Comparison of M|**MDI**|]<br>0.3|29.2|0.5<br>ation [0/1]<br>0.2|sing MEC course d<br>C.|
|Table 3||rse [0/1|(years)|registr|tions u<br> to MDI|
|||d cou|dent|]<br>ed at|alcula<br>uests|
||**ble**|lete|f stu|[0/1<br>ploy|s’ c<br> req|
||**Varia**|Comp|Age o|Male <br>Unem|Source: Author<br>municipality of|

31
Table 4: Summary statistics, all registrants in MDIC-requested courses

|**Variable**|**Mean**|**Std. Dev.**|**Min.**|**Max.**|**N**|
|---|---|---|---|---|---|
|Worker number of observations|30.55|4.183|18|35|335300|
|Male[0/1]|0.6|0.49|0|1|335300|
|Course Ofer|0.931|0.253|0|1|335300|
|Unemployment benefts registrant|0.185|0.388|0|1|335300|
|MDIC registrant|0.087|0.282|0|1|335300|
|Completed course[0/1]|0.397|0.489|0|1|335300|
|Course had admin. noncompleters [0/1]|0.597|0.49|0|1|335300|
|Employed [0/1]|0.565|0.496|0|1|10243541|
|Employed in an MDIC-requesting frm|0.041|0.199|0|1|10243541|
|Gross defated monthly earnings(R$)|1186.507|771.306|1|6139.095|5458730|
|ln( gross defated monthly earnings(R$))|6.897|0.643|0|8.722|5458730|
|Gross defated monthly wage rates|1251.957|787.062|1.009|6139.095|5458902|
|ln(gross defated monthly wage rates)|6.982|0.547|0.009|8.722|5458902|

Notes: Authors’ calculations using RAIS.

32
Table 5: Program employment effects, difference-in-differences estimates

|Outcome:|**Employed [0/1]**||
|---|---|---|
|Sample<br>All registrants|MDIC registrants<br>Unemp. Benefc.|All others|
|(1)|(2)<br>(3)|(4)|
|**Panel A: Em**|**ployment at any frm**||
|Post-course * completed<br>0.023***|0.031***<br>-0.042***|0.005|
|(0.003)|(0.006)<br>(0.006)|(0.003)|
|Mean of outcome<br>0.565|0.748<br>0.604|0.533|
|St. dev. of outcome<br>0.496|0.434<br>0.409|0.499|
|_R_<sup>2</sup><br>0.40|0.45<br>0.39|0.43|
|**Panel B: Employ**|**ment at requesting frms**||
|Post-course * completed<br>0.004***|0.020***<br>-0.003**|0.004***|
|(0.000)|(0.004)<br>(0.001)|(0.000)|
|Mean of outcome<br>0.041|0.237<br>0.009|0.025|
|St. dev. of outcome<br>0.198|0.425<br>0.094|0.156|
|_R_<sup>2</sup><br>0.76|0.82<br>0.42|0.69|
|**Panel C: Employm**|**ent at nonrequesting frms**||
|Post-course * completed<br>0.018***|0.010<br>-0.038***|0.001|
|(0.003)|(0.007)<br>(0.006)|(0.003)|
|Mean of outcome<br>0.524|0.511<br>0.595|0.508|
|St. dev. of outcome<br>0.499|0.5<br>0.491|0.5|
|_R_<sup>2</sup><br>0.42|0.60<br>0.39|0.44|
|_N_ (all panels)<br>10,243,541|912,053<br>1,872,837|7,458,651|

Notes: Table presents present difference-in-differences IV estimates of course completion. Heteroskedasticity-consistent robust standard errors two-way clustered by class and month*year reported in parentheses. All specifications include an unreported constant term and vectors of individual and month*year fixed effects. Significance indicated by: _∗ p < ._ 1, _∗∗ p < ._ 05, _∗∗∗ p < ._ 01.

33
Table 6: Testing for differential pre-course trends

|Outcome:||**Employed **|**[0/1]**||
|---|---|---|---|---|
|Sample|All registrants|MDIC registrants|Unemp. Benefc.|All others|
||(1)|(2)|(3)|(4)|
|Months relative to course|0.000928|-0.001781|-0.001771|-0.000028|
|* course ofer|(0.001117)|(0.001428)|(0.001851)|(0.000532)|
|_R_<sup>2</sup>|0.61|0.60|0.32|0.62|
|_N_|4,794,972|485,327|816,802|3,492,843|

Notes: Table presents estimates from the estimation of equation 3 in the text, adjusted to test parallel trends in pre-course employment differentially across those receiving the course offer and those not. Sample comprised of all periods prior to the start of registrants’ training course. The coefficient for [Months relative to course*offer] gives the differential slope term for those offered a course seat in the pre-course period. Heteroskedasticity-consistent robust standard errors two-way clustered by individual and month*year reported in parentheses. All specifications include an unreported constant term, a primary slope coefficient for months relative to the course, and vectors of individual and month*year fixed effects. Significance indicated by: _∗ p < ._ 1, _∗∗ p < ._ 05, _∗∗∗ p < ._ 01.

Table 7: First stage estimates, difference-in-differences IV

|Outcome:|**Post-**|**course period [0/1**|**] * Completed co**|**urse**|
|---|---|---|---|---|
|Sample|All registrants|MDIC registrants|Unemp. Benefc.|All others|
||(1)|(2)|(3)|(4)|
|Post-course period [0/1]|0.435062***|0.519792***|0.367063***|0.442523***|
|* course ofer|(0.003203)|(0.006737)|(0.007700)|(0.002292)|
|_R_<sup>2</sup>|0.57|0.61|0.54|0.57|
|_N_|10,243,541|912,053|1,872,837|7,458,651|

Notes: Table presents first-stage coefficients from the estimation of the endogenous variable [Post-course * Completed] in equation 1 as predicted by an indicator for being in the post-course period and having received a course offer. Heteroskedasticity-consistent robust standard errors two-way clustered by class and month*year reported in parentheses. All specifications include an unreported constant term and vectors of individual and month*year fixed effects. Significance indicated by: _∗ p < ._ 1, _∗∗ p < ._ 05, _∗∗∗ p < ._ 01.

34
Table 8: Program employment effects, difference-in-differences IV estimates

|Outcome:|**Employed [0/1]**||
|---|---|---|
|Sample<br>All registrants|MDIC registrants<br>Unemp. Benefc.|All others|
|(1)|(2)<br>(3)|(4)|
|**Panel A: Em**|**ployment at any frm**||
|Post-course * completed<br>0.086***|-0.029<br>0.146***|0.023*|
|(0.014)|(0.032)<br>(0.032)|(0.013)|
|Mean of outcome<br>0.565|0.748<br>0.604|0.533|
|St. dev. of outcome<br>0.496|0.434<br>0.409|0.499|
|_R_<sup>2</sup><br>0.41|0.45<br>0.37|0.43|
|**Panel B: Employ**|**ment at requesting frms**||
|Post-course * completed<br>0.009***|0.032<br>0.005|0.009***|
|(0.002)|(0.022)<br>(0.003)|(0.002)|
|Mean of outcome<br>0.041|0.237<br>0.009|0.025|
|St. dev. of outcome<br>0.198|0.425<br>0.094|0.156|
|_R_<sup>2</sup><br>0.76|0.82<br>0.42|0.69|
|**Panel C: Employm**|**ent at nonrequesting frms**||
|Post-course * completed<br>0.077***|-0.062*<br>0.140***|0.014|
|(0.014)|(0.036)<br>(0.032)|(0.013)|
|Mean of outcome<br>0.524|0.511<br>0.595|0.508|
|St. dev. of outcome<br>0.499|0.5<br>0.491|0.5|
|_R_<sup>2</sup><br>0.42|0.60<br>0.37|0.44|
|_N_<br>10,243,541|912,053<br>1,872,837|7,458,651|

Notes: Table presents second-stage coefficients capturing effects of course completion induced by receiving a course offer. Heteroskedasticity-consistent robust standard errors two-way clustered by class and month*year reported in parentheses. All specifications include an unreported constant term and vectors of individual and month*year fixed effects. Significance indicated by: _∗ p < ._ 1, _∗∗ p < ._ 05, _∗∗∗ p < ._ 01.

35
Table 9: Pronatec program employment effects, difference-in-differences IV estimates

|Outcome:||**Employed [0/1]**||
|---|---|---|---|
|Sample|All registrants|Unemp. Benefc.|All others|
||(1)|(2)|(3)|
|Post-course * completed|0.005|0.088***|-0.112***|
||(0.016)|(0.021)|(0.013)|
|Mean of outcome|0.561|0.602|0.558|
|St. dev. of outcome|0.496|0.489|0.497|
|_R_<sup>2</sup>|0.45|0.40|0.46|
|_N_|51,082,933|3,505,993|47,576,940|

Notes: Table presents second-stage coefficients capturing effects of course completion induced by receiving a course offer, estimated on the sample of all _Pronatec_ course registrants excluding MDIC-requested courses. Heteroskedasticity-consistent robust standard errors two-way clustered by class and month*year reported in parentheses. All specifications include an unreported constant term and vectors of individual and month*year fixed effects. Significance indicated by: _∗ p < ._ 1, _∗∗ p < ._ 05, _∗∗∗ p < ._ 01.

36
Table 10: Comparing demand-driven and overall program effects by subgroup

|**Estimated coef**|**cients**|
|---|---|
|Demand-driven program|Pronatec (all)|
|(1)|(2)|
|All sectors<br>0.086***|0.005|
|_Panel A: by occupational group_||
|Industrial Processes and Production<br>0.106**|0.001|
|Management and Business<br>0.093***|0.079*|
|Infrastructure<br>0.085***|0.064***|
|Information and Communication<br>0.153***|0.074|
|Environment, Health and Education<br>0.088**|0.117***|
|Food Production<br>0.185*|0.171***|
|Natural Resources, Culture and Tourism<br>0.118**|0.056*|
|_Panel B: by occupational skill/wage level_||
|1 (Low)<br>0.080**|0.046***|
|2<br>0.090***|0.057***|
|3<br>0.086***|-0.043|
|4<br>0.127***|-0.328***|
|5 (High)<br>-0.198*|-0.771***|
|_Panel C: by region_||
|North<br>0.105***|0.017|
|Northeast<br>0.078***|0.016|
|Southeast<br>0.090***|0.014|
|South<br>0.043*|-0.024|
|Midwest<br>0.045**|0.027|
|_Panel D: by worker education level_||
|Primary Incomplete<br>0.056|-0.043|
|Primary<br>0.236***|0.107***|
|Middle<br>0.094***|0.026|
|Secondary<br>0.074***|0.052***|
|Post-secondary<br>0.036|0.033|

Notes: Columns 1 and 2 present difference-in-differences IV estimates of course completion, estimated separately by course or trainee characteristics in firm-requested and broader _Pronatec_ courses. Significance for coefficients presented in Columns 1 and 2 indicated by: _∗ p < ._ 1, _∗∗ p < ._ 05, _∗∗∗ p < ._ 01.

37
Table 11: Alternative crosssectional IV: First-stage estimates of completion based on course

|Outcome:||**Completed**|**course [0/1]**|
|---|---|---|---|
|Specifcation:|Month FE|Class FE|Class FE<br>+ educ<br>Class FE<br>+ educ + _Et−_1_,_3_,_6_,_9|
||(1)|(2)|(2)<br>(3)|
|Course ofer|0.427<br>(0.000)***|0.398<br>(0.002)***|0.397<br>0.398<br>(0.002)***<br>(0.002)***|
|Worker education (years)|||0.009<br>0.009<br>(0.001)***<br>(0.001)***|
|Employment 6 mos.|||0.015|
|before course start|||(0.001)***|
|Month fxed efects|Y|N|N<br>N|
|Class fxed efects|N|Y|Y<br>Y|
|_N_|335,300|334,954|334,954<br>334,249|
|_R_<sup>2</sup>|0.049|0.229|0.231<br>0.231|

Notes: Table presents coefficients from the crosssectional first-stage regression estimating completion with course offer status across specifications indicated in the column headings. Heteroskedasticity-consistent robust standard errors clustered by course reported in parentheses. All specifications include an unreported constant term. Significance indicated by: _∗ p < ._ 1, _∗∗ p < ._ 05, _∗∗∗ p < ._ 01.

38
Table 12: Alternative crosssectional IV: Second-stage coefficient estimates by time period – relative to course start All MDIC registrants

|Outcome:||**Empl**|**oyed [0/1]**||
|---|---|---|---|---|
|Specifcation:|Month FE|Class FE|Class FE<br>+ educ|Class FE<br>+ educ + _Et−_1_,_3_,_6_,_9|
||(1)|(2)|(3)|(4)|
|t+6 (course ends)|0.081***|0.070***|0.067***|0.110***|
|t+7|0.071***|0.060***|0.058***|0.098***|
|t+8|0.073***|0.069***|0.066***|0.104***|
|t+9|0.068***|0.059***|0.057***|0.092***|
|t+10|0.069***|0.059***|0.056***|0.090***|
|t+11|0.058***|0.051***|0.048***|0.078***|
|t+12|0.058***|0.051***|0.047***|0.077***|
|t+13|0.053***|0.047***|0.043***|0.072***|
|t+14|0.048***|0.041***|0.038***|0.065***|
|t+15|0.052***|0.050***|0.046***|0.070***|
|t+16|0.055***|0.057***|0.053***|0.075***|
|t+17|0.050***|0.046***|0.043***|0.064***|

Notes: Table presents second-stage coefficients from the estimation of equation 5 in the text. Each row comes from a separate cross-sectional estimation of employment at the indicated month relative to course start. Specification variants are indicated in the column headings. Heteroskedasticity-consistent robust standard errors two-way clustered by course and month*year reported in parentheses. All specifications include an unreported constant term. Significance indicated by: _∗ p < ._ 1, _∗∗ p < ._ 05, _∗∗∗ p < ._ 01.

39
Table 13: Alternative crosssectional IV: Second-stage coefficient estimates by time period – relative to course start UB registrants only

|Outcome:||**Empl**|**oyed [0/1]**||
|---|---|---|---|---|
|Specifcation:|Month FE|Class FE|Class FE<br>+ educ|Class FE<br>+ educ + _Et−_1_,_3_,_6_,_9|
||(1)|(2)|(3)|(4)|
|t+6 (course ends)|0.087***|0.093***|0.092***|0.094***|
|t+7|0.085***|0.094***|0.092***|0.093***|
|t+8|0.097***|0.113***|0.111***|0.112***|
|t+9|0.097***|0.108***|0.107***|0.105***|
|t+10|0.110***|0.114***|0.113***|0.113***|
|t+11|0.089***|0.094***|0.092***|0.089***|
|t+12|0.085***|0.084***|0.082***|0.081***|
|t+13|0.075***|0.077***|0.074***|0.072***|
|t+14|0.069***|0.061***|0.059**|0.058**|
|t+15|0.068***|0.070***|0.068***|0.067***|
|t+16|0.071***|0.075***|0.072***|0.071***|
|t+17|0.061***|0.061**|0.058**|0.057**|

Notes: Table presents second-stage coefficients from the estimation of equation 5 in the text for the sample of unemployment beneficiary registrants. Each row comes from a separate cross-sectional estimation of employment at the indicated month relative to course start. Specification variants are indicated in the column headings. Heteroskedasticity-consistent robust standard errors two-way clustered by course and month*year reported in parentheses. All specifications include an unreported constant term. Significance indicated by: _∗ p < ._ 1, _∗∗ p < ._ 05, _∗∗∗ p < ._ 01.

40
|employment?|**h, 2013-2015**|Main program<br>(4)|0.289|0.138|0.061<br>e reference distributions|
|---|---|---|---|---|---|
|aggregate|**t. growt**|program<br>|||aggregat|
|or fow of|**tion:**<br>**gate emp**|nd-driven <br>(3)|0.219|0.194|0.057<br>aining and<br>ings.|
|the stock|**distribu**<br>**Aggre**|Dema|||ogram tr<br>row head|
|ining to|**ference **<br>**on**|program<br>(2)|.033|.035|.004<br>cross pr<br>cated in|
|tra|**Re**<br>**cti**|ain|0|0|0<br>D) a<br> indi|
|m match|**crossse**|ram<br>M|||ns (SS<br>riables|
|n progra|**empt. **|iven prog<br>1)|050|057|006<br> deviatio<br>gorical va|
|the demand-drive|**2013**|Demand-dr<br>ross:<br>(|on<br>0.|0.|n<br>0.<br>e sum of squared<br>dings) for the cate|
|: Did||gram:<br>ion ac|cupati||upatio<br>ents th<br>n hea|
|14||pro<br>ut|oc||cc<br>es<br>um|
|Table||Focal <br>Distrib|3-digit|Dtate|State-o<br>Notes: Table pr<br>(indicated in col|

41
|mates|All others<br>(4)<br>0.011<br>(0.014)<br>0.005***<br>(0.001)|0.533<br>0.499<br>0.42<br>5,985,104<br>-0.136***<br>(0.016)<br>0.012***<br>(0.001)<br>0.025<br>0.156<br>0.46<br>42,920,272<br>consistent robust<br>e an unreported<br>_∗∗p < ._05, _∗∗∗_|
|---|---|---|
|Table 15: Diferential efects by post-program employment growth, diference-in-diferences IV esti|Outcome:<br>**Employed [0/1]**<br>Sample<br>All registrants<br>MDIC registrants<br>Unemp. Benefc.<br>(1)<br>(2)<br>(3)<br>**Panel A: Demand-driven segment**<br>Main efect<br>0.075***<br>-0.040<br>0.155***<br>(0.015)<br>(0.034)<br>(0.034)<br>Main * standardized employment growth share<br>0.005***<br>0.007*<br>-0.000<br>(0.001)<br>(0.004)<br>(0.002)|Mean of outcome<br>0.565<br>0.748<br>0.604<br>St. dev. of outcome<br>0.496<br>0.434<br>0.409<br>_R_2<br>0.40<br>0.44<br>0.37<br>_N_<br>8,212,030<br>817,655<br>1,409,271<br>**Panel B: All other courses**<br>Main efect<br>-0.005<br>0.105***<br>(0.018)<br>(0.023)<br>Main * standardized employment growth share<br>0.011***<br>-0.000<br>(0.001)<br>(0.001)<br>Mean of outcome<br>0.041<br>0.009<br>St. dev. of outcome<br>0.198<br>0.094<br>_R_2<br>0.45<br>0.39<br>_N_<br>45,919,339<br>2,752,395<br>Notes: Table presents present diference-in-diferences IV estimates of course completion. Heteroskedasticity-<br>standard errors two-way clustered by class and month*year reported in parentheses. All specifcations includ<br>constant term and vectors of individual and month*year fxed efects. Signifcance indicated by: _∗p < ._1, <br>_p < ._01.|

42
# null ,”
Figure 2: Change in employment relative to course start: completers and non-completers <!-- Start of picture text --> All registrants<br>Course<br>-18 -16 -14 -12 -10 -8 -6 -4 -2 0 2 4 6 8 10 12 14 16 18<br>Months relative to start of employment training<br>Non-completers Completers<br>.41<br>.36<br>.31<br>.26<br>.21<br>.16<br>.11<br>Parameter estimate with 95% CI<br>.06<br>.01<br>-.04<br><!-- End of picture text --> Notes: Figure depicts the set of coefficients for course completers and noncompleters corresponding to ( _αk_ + _βk_ ) and ( _αk_ ), respectively, from equation 2 estimated for employment [0/1]. 95% confidence intervals constructed from heteroskedasticity-consistent standard errors clustered by individual and month.

44
Figure 3: Change in ln(real monthly wage rate) relative to course start: completers and non-completers <!-- Start of picture text --> All registrants<br>Course<br>-18 -16 -14 -12 -10 -8 -6 -4 -2 0 2 4 6 8 10 12 14 16 18<br>Months relative to start of employment training<br>Non-completers Completers<br>.06<br>.01<br>Parameter estimate with 95% CI<br>-.04<br><!-- End of picture text --> Notes: Figure depicts the set of coefficients for course completers and non-completers corresponding to ( _αk_ + _βk_ ) and ( _αk_ ), respectively, from equation 2 estimated for ln(real monthly earnings). Sample implicitly comprised of person-months with positive wage earnings. 95% confidence intervals constructed from heteroskedasticity-consistent standard errors clustered by individual and month.

45
Figure 4: Change in employment relative to course start: completers and non-completers, MDIC registrants <!-- Start of picture text --> MDIC registrants<br>Course<br>-18 -16 -14 -12 -10 -8 -6 -4 -2 0 2 4 6 8 10 12 14 16 18<br>Months relative to start of employment training<br>Non-completers Completers<br>.11<br>.06<br>.01<br>-.04<br>Parameter estimate with 95% CI<br>-.09<br>-.14<br><!-- End of picture text --> Notes: Figure depicts the set of coefficients for course completers and noncompleters corresponding to ( _αk_ + _βk_ ) and ( _αk_ ), respectively, from equation 2 estimated for employment [0/1]. 95% confidence intervals constructed from heteroskedasticity-consistent standard errors clustered by individual and month.

46
Figure 5: Change in employment relative to course start: completers and non-completers, UB registrants <!-- Start of picture text --> UB registrants<br>Course<br>-18 -16 -14 -12 -10 -8 -6 -4 -2 0 2 4 6 8 10 12 14 16 18<br>Months relative to start of employment training<br>Non-completers Completers<br>1.2<br>1.1<br>1<br>.9<br>.8<br>.7<br>.6<br>.5<br>.4<br>.3<br>Parameter estimate with 95% CI .2<br>.1<br>0<br>-.1<br><!-- End of picture text --> Notes: Figure depicts the set of coefficients for course completers and noncompleters corresponding to ( _αk_ + _βk_ ) and ( _αk_ ), respectively, from equation 2 estimated for employment [0/1]. 95% confidence intervals constructed from heteroskedasticity-consistent standard errors clustered by individual and month.

47
Figure 6: Change in employment relative to course start: completers and non-completers, all other registrants <!-- Start of picture text --> All other registrants<br>Course<br>-18 -16 -14 -12 -10 -8 -6 -4 -2 0 2 4 6 8 10 12 14 16 18<br>Months relative to start of employment training<br>Non-completers Completers<br>.37<br>.32<br>.27<br>.22<br>.17<br>.12<br>Parameter estimate with 95% CI .07<br>.02<br>-.03<br><!-- End of picture text --> Notes: Figure depicts the set of coefficients for course completers and noncompleters corresponding to ( _αk_ + _βk_ ) and ( _αk_ ), respectively, from equation 2 estimated for employment [0/1]. 95% confidence intervals constructed from heteroskedasticity-consistent standard errors clustered by individual and month.

48
Figure 7: Change in employment relative to course start: completers and non-completers, MDIC registrants at MDIC firms <!-- Start of picture text --> MDIC reg. at requesting firms<br>Course<br>-18 -16 -14 -12 -10 -8 -6 -4 -2 0 2 4 6 8 10 12 14 16 18<br>Months relative to start of employment training<br>Non-completers Completers<br>.05<br>0<br>-.05<br>-.1<br>Parameter estimate with 95% CI<br>-.15<br>-.2<br><!-- End of picture text --> Notes: Figure depicts the set of coefficients for course completers and noncompleters corresponding to ( _αk_ + _βk_ ) and ( _αk_ ), respectively, from equation 2 estimated for employment [0/1]. 95% confidence intervals constructed from heteroskedasticity-consistent standard errors clustered by individual and month.

49
Figure 8: Change in employment relative to course start: completers and non-completers, UB registrants at non-MDIC firms <!-- Start of picture text --> UB reg. at nonrequesting firms<br>Course<br>-18 -16 -14 -12 -10 -8 -6 -4 -2 0 2 4 6 8 10 12 14 16 18<br>Months relative to start of employment training<br>Non-completers Completers<br>1.2<br>1.1<br>1<br>.9<br>.8<br>.7<br>.6<br>.5<br>.4<br>.3<br>Parameter estimate with 95% CI .2<br>.1<br>0<br>-.1<br><!-- End of picture text --> Notes: Figure depicts the set of coefficients for course completers and noncompleters corresponding to ( _αk_ + _βk_ ) and ( _αk_ ), respectively, from equation 2 estimated for employment [0/1]. 95% confidence intervals constructed from heteroskedasticity-consistent standard errors clustered by individual and month.

50
Figure 9: Change in employment relative to course start: completers and non-completers, all other registrants at MDIC firms <!-- Start of picture text --> All other reg. at requesting firms<br>Course<br>-18 -16 -14 -12 -10 -8 -6 -4 -2 0 2 4 6 8 10 12 14 16 18<br>Months relative to start of employment training<br>Non-completers Completers<br>.01<br>0<br>Parameter estimate with 95% CI<br>-.01<br><!-- End of picture text --> Notes: Figure depicts the set of coefficients for course completers and noncompleters corresponding to ( _αk_ + _βk_ ) and ( _αk_ ), respectively, from equation 2 estimated for employment [0/1]. 95% confidence intervals constructed from heteroskedasticity-consistent standard errors clustered by individual and month.

51
Figure 10: Change in employment relative to course start: completers and non-completers, all other registrants at non-MDIC firms <!-- Start of picture text --> All other reg. at nonrequesting firms<br>Course<br>-18 -16 -14 -12 -10 -8 -6 -4 -2 0 2 4 6 8 10 12 14 16 18<br>Months relative to start of employment training<br>Non-completers Completers<br>.37<br>.32<br>.27<br>.22<br>.17<br>.12<br>Parameter estimate with 95% CI .07<br>.02<br>-.03<br><!-- End of picture text --> Notes: Figure depicts the set of coefficients for course completers and noncompleters corresponding to ( _αk_ + _βk_ ) and ( _αk_ ), respectively, from equation 2 estimated for employment [0/1]. 95% confidence intervals constructed from heteroskedasticity-consistent standard errors clustered by individual and month.

52
Appendix: For Web Publication Only

# **A1 Cost calculations**

Using parameters from the data and estimated employment effects, we calculate the net present value (NPV) of the training course based on increased employment earnings. We describe the funding/cost structure of the program below, and then present calculations for the representative course and trainee. It is important to note that these calculations do not include administrative costs nor impacts on training markets, including possible crowding out of alternate training providers. Furthermore, the calculations rely on several assumptions regarding interest and depreciation rates, as well as the rate at which formal employment displaces informal earnings.

Pronatec courses were financed by the federal government on a student-hour basis. For 2014, cost per student-hour was R$10.00 for in-person courses. Course providers were then reimbursed for the student-hours of “confirmed students.” These were students who: (a) attended at least 20% of the course hours or (b) attended 25% of the course hours in the first four months of the course.<sup>10</sup> Course providers were reimbursed according to the full number of hours of the course for all students who reached this minimum attendance threshold, independent of whether the student completed those hours or dropped out before the end of the course.

The average course in the demand-driven program consists of 200 hours of classroom time. Across all courses, this results in an average cost of R$2,000 per trainee (approximately 1,780 in real 2012 Reais). However, because some registrants did not incur costs as described above, the per-registrant cost is slightly smaller than the amount above (80 – percent of course registrants are counted in the calculation of course costs) resulting in a per registrant cost figure of approximately R$1,470. In this calculation, we assume that the benefits of the program are captured entirely in earnings from increased employment

> 10See article 64 of Portaria No. 168/2013, available at `http://pronatec.mec.gov.br/images/stories/ pdf/port_168_070313.pdf` .

53
Appendix: For Web Publication Only rather than increased wage rates. The estimates of employment effects used in conjunction with the median monthly wage rate of approximately R$1,380 to estimate the nominal monthly earnings benefit of the program of approximately R$119 per month assuming an 8.6 percentage-point employment effect. However, since these benefits are realized only by course completers, the per-registrant benefit is around 40% of this figure, or approximately R$46 per month.

This figure, however, does not account for the displacement effect of increased formal employment, which will reduce the net benefit depending on the degree to which trainees would have otherwise been informally employed. We first assume that the formal net wage premium is 20 percent, and then present calculations based on two scenarios. The first, more conservative scenario assumes that all those who gained formal employment due to the program would have been otherwise informally employed, so displaced earnings are 80% of the gained employment earnings. The second scenario assumes that the program displaces informal employment among 75% of trainees, the remaining 25% of whom would have been – unemployed resulting in a net earnings displacement of 60%. Assuming a five percent annual interest rate and also five percent annual real depreciation of skills, we find that under 60% earnings displacement the per-registrant NPV of the program is R$2,189, which after course fiscal costs yields a net benefit of R$638. Assuming displaced informal earnings would have been 80% of formal earnings, the program yields a NPV of R$1,094 per registrant, resulting in a net benefit of -R$456 after course costs.

54
Appendix: For Web Publication Only

# **A2 Appendix Tables and Figures**

Appendix Table 1: Summary statistics, MDIC registrants

|**Variable**|**Mean**|**Std. Dev.**|**Min.**|**Max.**|**N**|
|---|---|---|---|---|---|
|Worker number of observations|31.131|4.283|18|35|29297|
|Male[0/1]|0.781|0.414|0|1|29297|
|Course Ofer|0.96|0.195|0|1|29297|
|Unemployment benefts registrant|0|0|0|0|29297|
|MDIC registrant|1|0|1|1|29297|
|Completed course[0/1]|0.493|0.5|0|1|29297|
|Course had admin. noncompleters [0/1]|0.341|0.474|0|1|29297|
|Employed [0/1]|0.748|0.434|0|1|912053|
|Employed in an MDIC-requesting frm|0.237|0.425|0|1|912053|
|Gross defated monthly earnings(R$)|1491.79|879.04|1.038|6138.825|659514|
|ln(gross defated monthly earnings(R$))|7.148|0.593|0.038|8.722|659514|
|Gross defated monthly wage rates|1530.628|882.11|1.218|6138.825|659520|
|ln(gross defated monthly wage rates)|7.191|0.537|0.197|8.722|659520|
|Notes: Authors’ calculations using RAIS.||||||

Appendix Table 2: Summary statistics, Unemployment benefits registrants

|**Variable**|**Mean**|**Std. Dev.**|**Min.**|**Max.**|**N**|
|---|---|---|---|---|---|
|Worker number of observations|30.209|3.76|18|35|61995|
|Male[0/1]|0.61|0.488|0|1|61995|
|Course Ofer|0.832|0.374|0|1|61995|
|Unemployment benefts registrant|1|0|1|1|61995|
|MDIC registrant|0|0|0|0|61995|
|Completed course[0/1]|0.3|0.458|0|1|61995|
|Course had admin. noncompleters [0/1]|0.822|0.382|0|1|61995|
|Employed [0/1]|0.604|0.489|0|1|1872837|
|Employed in an MDIC-requesting frm|0.009|0.096|0|1|1872837|
|Gross defated monthly earnings(R$)|1134.875|666.16|1.019|6139.095|1091600|
|ln( gross defated monthly earnings(R$))|6.881|0.601|0.019|8.722|1091600|
|Gross defated monthly wage rates|1213.404|694.817|1.01|6139.095|1091624|
|ln(gross defated monthly wage rates)|6.982|0.481|0.01|8.722|1091624|

Notes: Authors’ calculations using RAIS.

55
Appendix: For Web Publication Only Appendix Table 3: Summary statistics, All other registrants

|**Variable**|**Mean**|**Std. Dev.**|**Min.**|**Max.**|**N**|
|---|---|---|---|---|---|
|Worker number of observations|30.567|4.264|18|35|244008|
|Male[0/1]|0.576|0.494|0|1|244008|
|Course Ofer|0.953|0.212|0|1|244008|
|Unemployment benefts registrant|0|0|0|0|244008|
|MDIC registrant|0|0|0|0|244008|
|Completed course[0/1]|0.411|0.492|0|1|244008|
|Course had admin. noncompleters [0/1]|0.571|0.495|0|1|244008|
|Employed [0/1]|0.533|0.499|0|1|7458651|
|Employed in an MDIC-requesting frm|0.025|0.157|0|1|7458651|
|Gross defated monthly earnings(R$)|1129.573|714.946|1|5209.774|3690130|
|ln(gross defated monthly earnings(R$))|6.849|0.645|0|8.558|3690130|
|Gross defated monthly wage rates|1192.811|724.908|1.009|5209.774|3690272|
|ln(gross defated monthly wage rates)|6.937|0.547|0.009|8.558|3690272|

Notes: Authors’ calculations using RAIS.

56
Appendix: For Web Publication Only

|estimates<br>**s**<br>All others<br>(4)<br>0.043<br>(0.004)***<br>0.042<br>(0.004)***<br>0.046<br>(0.004)***<br>0.048<br>(0.004)***<br>0.038<br>(0.003)***<br>-0.002<br>(0.003)<br>0.035<br>(0.004)***<br>0.056<br>(0.004)***<br>0.073<br>(0.005)***<br>0.079<br>(0.004)***<br>Y<br>Y<br>0.533<br>0.499<br>0.43<br>7,458,651<br>leters versus non-completers<br>t) for the sample group indi-<br>red by class and month*year<br>vidual and month*year fxed|
|---|
|t efects of worker training: selected _β_ coefcient <br>**Estimation of employment across**<br>**course completers and non-completer**<br>**Employed [0/1]**<br>All registrants<br>MDIC registrants<br>Unemp. Benefc.<br>(1)<br>(2)<br>(3)<br>0.014<br>0.041<br>0.135<br>(0.006)**<br>(0.010)***<br>(0.014)***<br>0.008<br>0.04<br>0.119<br>(0.006)<br>(0.010)***<br>(0.009)***<br>0.002<br>0.039<br>0.105<br>(0.006)<br>(0.010)***<br>(0.010)***<br>0.001<br>0.028<br>0.105<br>(0.007)<br>(0.010)***<br>(0.011)***<br>0.03<br>0.012<br>0.094<br>(0.006)***<br>(0.009)<br>(0.018)***<br>-0.014<br>0.045<br>0.011<br>(0.006)**<br>(0.012)***<br>(0.011)<br>0.018<br>0.059<br>0.062<br>(0.006)***<br>(0.011)***<br>(0.011)***<br>0.037<br>0.068<br>0.083<br>(0.005)***<br>(0.012)***<br>(0.010)***<br>0.052<br>0.073<br>0.094<br>(0.005)***<br>(0.013)***<br>(0.010)***<br>0.059<br>0.074<br>0.098<br>(0.005)***<br>(0.013)***<br>(0.010)***<br>Y<br>Y<br>Y<br>Y<br>Y<br>Y<br>0.565<br>0.748<br>0.604<br>0.496<br>0.434<br>0.409<br>0.41<br>0.45<br>0.44<br>10,243,541<br>912,053<br>1,872,837<br>ng the diference in outcomes between course comp<br>urse completion (_β_ vector in equation 2 in the tex<br>consistent robust standard errors two-way cluste<br> an unreported constant term and vectors of indi<br>_._05, _∗∗∗p < ._01.|
|Appendix Table 4: Employmen<br>Outcome:<br>Sample<br>18 months prior to course start (t=-18)<br>12 months prior to course start (t=-12)<br>9 months prior to course start (t=-9)<br>6 months prior to course start (t=-6)<br>1 month prior to course start (t=-1)<br>At end of course (t=+6)<br>3 months after course (t=+9)<br>6 months after course (t=+12)<br>9 months after course (t=+15)<br>12 months after course (t=+18)<br>Individual fxed efects<br>Year * Month fxed efects<br>Mean of outcome<br>St. dev. of outcome<br>_R_2<br>_N_<br>Notes: Table presents selected coefcients measuri<br>at diferent time periods relative to the start of co<br>cated in the column headings. Heteroskedasticity-<br>reported in parentheses. All specifcations include <br>efects. Signifcance indicated by: _∗p < ._1, _∗∗p <_|

57
Appendix: For Web Publication Only

|lected _β_ coefcient estimates|**uesting status**<br>**ed in other frm [0/1]**<br>Unemp. Benefc.<br>All others<br>(5)<br>(6)<br>0.132<br>0.047<br>(0.014)***<br>(0.004)***<br>0.114<br>0.043<br>(0.009)***<br>(0.004)***<br>0.1<br>0.047<br>(0.011)***<br>(0.004)***<br>0.1<br>0.048<br>(0.012)***<br>(0.004)***<br>0.092<br>0.038<br>(0.018)***<br>(0.003)***<br>0.011<br>-0.003<br>(0.011)<br>(0.003)<br>0.060<br>0.031<br>(0.011)***<br>(0.004)***<br>0.082<br>0.052<br>(0.010)***<br>(0.004)***<br>0.093<br>0.069<br>(0.010)***<br>(0.005)***<br>0.097<br>0.075<br>(0.010)***<br>(0.004)***<br>Y<br>Y<br>Y<br>Y<br>0.595<br>0.508<br>0.491<br>0.5<br>0.44<br>0.44<br>1,872,837<br>7,458,651<br>ompleters versus non-completers<br>r the sample group indicated in<br>y class and month*year reported<br>al and month*year fxed efects.|
|---|---|
|efects due to MDIC-requestors and outside employers: se|**Estimation of employment across**<br>**course completers and non-completers, by employer req**<br>**Employed in requesting frm [0/1]**<br>**Employ**<br>MDIC<br>UB<br>All others<br>MDIC<br><br>(1)<br>(2)<br>(3)<br>(4)<br>0.005<br>0.003<br>-0.003<br>0.036<br>(0.008)<br>(0.001)**<br>(0.000)***<br>(0.010)***<br>0.005<br>0.005<br>-0.001<br>0.036<br>(0.007)<br>(0.001)***<br>(0.000)<br>(0.008)***<br>0.006<br>0.005<br>_<_0.001<br>0.033<br>(0.005)<br>(0.001)***<br>(0.000)<br>(0.008)***<br>0.004<br>0.005<br>_<_0.001<br>0.024<br>(0.005)<br>(0.001)***<br>(0.000)<br>(0.008)***<br>_<_0.001<br>0.002<br>0.001<br>0.012<br>(0.006)<br>(0.000)**<br>(0.000)**<br>(0.005)**<br>0.023<br>_<_0.001<br>0.001<br>0.022<br>(0.005)***<br>(0.000)<br>(0.000)**<br>(0.009)**<br>0.023<br>0.002<br>0.004<br>0.036<br>(0.006)***<br>(0.000)<br>(0.000)***<br>(0.010)***<br>0.026<br>0.001<br>0.004<br>0.042<br>(0.007)***<br>(0.001)<br>(0.000)***<br>(0.010)***<br>0.027<br>0.001<br>0.004<br>0.046<br>(0.009)***<br>(0.001)<br>(0.000)***<br>(0.012)***<br>0.026<br>0.001<br>0.004<br>0.048<br>(0.010)**<br>(0.001)<br>(0.000)***<br>(0.010)***<br>Y<br>Individual fxed efects<br>Y<br>Y<br>Y<br>Year * Month fxed efects<br>Y<br>Y<br>0.237<br>0.009<br>0.025<br>0.511<br>0.425<br>0.096<br>0.157<br>0.5<br>0.82<br>0.42<br>0.69<br>0.60<br>912,053<br>1,872,837<br>7,458,651<br>912,053<br>ents measuring the diference in outcomes between course c<br>the start of course completion (_β_ vector in equation 2) fo<br>city-consistent robust standard errors two-way clustered b<br>lude an unreported constant term and vectors of individu<br>_∗p < ._05, _∗∗∗p < ._01.|
|Appendix Table 5: Employment|Outcome:<br>Sample<br>18 months prior to course start (t=-18)<br>12 months prior to course start (t=-12)<br>9 months prior to course start (t=-9)<br>6 months prior to course start (t=-6)<br>1 month prior to course start (t=-1)<br>At end of course (t=+6)<br>3 months after course (t=+9)<br>6 months after course (t=+12)<br>9 months after course (t=+15)<br>12 months after course (t=+18)<br>Individual fxed efects<br>Year * Month fxed efects<br>Mean of outcome<br>St. dev. of outcome<br>_R_2<br>_N_<br>Notes: Table presents selected coefci<br>at diferent time periods relative to <br>the column headings. Heteroskedasti<br>in parentheses. All specifcations inc<br>Signifcance indicated by: _∗p < ._1, _∗_|

58
Appendix: For Web Publication Only Appendix Table 6: Testing selection of UB registrants on observables across programs

|Outcome:|**UB registrant took frm-requested course[0/1]**|
|---|---|
|Specifcation:|Base<br>Incl. wages<br>(1)<br>(2)|
|Employed 1 month before course [0/1]|-0.025***<br>-0.022**<br>(0.005)<br>(0.007)|
|Employed 3 months before course [0/1]|0.012*<br>0.025**<br>(0.007)<br>(0.009)|
|Employed 6 months before course [0/1]|-0.047***<br>-0.029<br>(0.012)<br>(0.015)|
|ln(wage) 1 month before course|-0.006*<br>(0.003)|
|ln(wage) 3 months before course|0.004<br>(0.003)|
|ln(wage) 6 months before course|0.011*<br>(0.005)|
|Education (years)|0.012***<br>(0.001)|
|Class fxed efects|Y<br>Y|
|_N_|167,706<br>167,706|
|_R_<sup>2</sup>|0.006<br>0.0065|

Notes: Table presents coefficients from the estimation of an indicator for being in a firm-requested course among all UB registrants of Pronatec and Pronatec-MDIC. Heteroskedasticity-consistent robust standard errors clustered by class reported in parentheses. All specifications include an unreported constant term. Significance indicated by: _∗ p < ._ 1, _∗∗ p < ._ 05, _∗∗∗ p < ._ 01.

59