metadata: authors: 0: author: Alzua Soyolmaa Batbekh,Maria Laura
1: author: Batchuluun,Altantsetseg
2: author: Dalkhjav,Bayarmaa
3: author: Galdo,Jose
count: Mongolia
docna: 0: docna: Demand-Driven Youth Training Programs :
            Experimental Evidence from Mongolia
docty: Policy Research Working Paper
subtopic: Vocational Education & Technical Training,Vocational & Technical Education,Labor Markets,Rural Labor Markets,Employment and Unemployment
teratopic: Education,Rural Development,Social Protections and Labor
abstracts: cdata!: The effectiveness of a demand-driven
            vocational-training program for disadvantaged youth in
            Ulaanbaatar, the capital city of Mongolia is assessed
            through a randomized controlled trial. Mongolia, a
            transitional country whose economic structure shifted from a
            Communist, centrally planned economy to a free-market
            economy over a relatively short period, offers a new setting
            in which to test the effectiveness of market-based
            active-labor-market policies. Results show short-term
            positive impacts on self-employment and skills match, while
            positive but uncertain effects emerge for employment and
            earnings. Substantial heterogeneity emerges as relatively
            older, richer, and better-educated individuals drive these
            positive effects. A second intervention, in which
            participants were randomly assigned to receive newsletters
            with information on market returns to vocational training,
            shows statistically meaningful effects on the length of
            exposure to the program (i.e., number of training days
            attended). These positive impacts, however, do not lead to
            higher employment or greater earnings.
geo_reg_key: 517198,517191,119225
repnme: repnme: Demand-Driven Youth Training Programs :
            Experimental Evidence from Mongolia
display_title: Demand-Driven Youth Training
            Programs : Experimental Evidence from Mongolia
listing_relative_url: /research/2020/04/31999780/demand-driven-youth-training-programs-experimental-evidence-mongolia
topicv3: Education, Skills Development and Labor
            Market m1326135 1484,Labor Markets m1326488 855,Vocational Education and Technical
            Training m1326570 1573,Vocational Education and Training m1326810 483,Self-Employment m1504000 2698
publishtoextweb_dt: 2020-04-30T00:00:00Z
historic_topic: Education,Rural Development,Social Protections and Labor
origu: DEC-Knowledge and Strategy (DECKS)
no_of_pages: 60
author_keys: 0: author_key: 1721219
1: author_key: 1197107
model_extractions: Policy Research Working Paper 9226 Demand-Driven Youth Training Programs Experimental Evidence from Mongolia _Maria Laura Alzúa Soyolmaa Batbekh Altantsetseg Batchuluun Bayarmaa Dalkhjav José Galdo_ Development Economics Knowledge and Strategy Team April 2020
Policy Research Working Paper 9226

# **Abstract**

The effectiveness of a demand-driven vocational-training program for disadvantaged youth in Ulaanbaatar, the capital city of Mongolia is assessed through a randomized controlled trial. Mongolia, a transitional country whose economic structure shifted from a Communist, centrally planned economy to a free-market economy over a relatively short period, offers a new setting in which to test the effectiveness of market-based active-labor-market policies. Results show short-term positive impacts on self-employment and skills match, while positive but uncertain effects emerge for employment and earnings. Substantial heterogeneity emerges as relatively older, richer, and better-educated individuals drive these positive effects. A second intervention, in which participants were randomly assigned to receive newsletters with information on market returns to vocational training, shows statistically meaningful effects on the length of exposure to the program (i.e., number of training days attended). These positive impacts, however, do not lead to higher employment or greater earnings.

This paper is a product of the Knowledge and Strategy Team, Development Economics. It is part of a larger effort by the World Bank to provide open access to its research and make a contribution to development policy discussions around the world. Policy Research Working Papers are also posted on the Web at http://www.worldbank.org/prwp. The authors may be contacted at malzua@depeco.econo.unlp.edu.ar.

_The Policy Research Working Paper Series disseminates the findings of work in progress to encourage the exchange of ideas about development issues. An objective of the series is to get the findings out quickly, even if the presentations are less than fully polished. The papers carry the names of the authors and should be cited accordingly. The findings, interpretations, and conclusions expressed in this paper are entirely those of the authors. They do not necessarily represent the views of the International Bank for Reconstruction and Development/World Bank and its affiliated organizations, or those of the Executive Directors of the World Bank or the governments they represent._ Produced by the Research Support Team
# Demand-Driven Youth Training Programs: Experimental Evidence

# from Mongolia

_Maria Laura Alzúa Soyolmaa Batbekh, Altantsetseg Batchuluun, Bayarmaa Dalkhjav,and José Galdo,_<sup>_1_</sup>

**JEL classification codes:** J18, J08, J24, J38, C93

**Keywords:** vocational-training programs, labor market, randomized controlled trial, employment, earnings, job quality.

> 1 María Laura Alzua (corresponding author) is Deputy Director at CEDLAS, Universidad Nacional de La Plata, Buenos Aires, Argentina and Research Director at the Partnership for Economic Development, her email address is malzua@depeco.econo.unlp.edu.ar. Soyolmaa Batbekh is an Associate Professor at the National University of Mongolia, Ulaanbaatar, Mongolia; her email address is soyolmaa@num.edu.mn. Altantsetseg Batchuluun is an Associate Professor at the National University of Mongolia, Ulaanbaatar, Mongolia; her email address is altantsetseg.b@num.edu.mn. Bayarmaa Dalkhjav is a lecturer at the National University of Mongolia, Ulaanbaatar, Mongolia; her email address is dbayarmaa@num.edu.mn. José Galdo is an Associate Professor at Carleton University, Ottawa, Canada; his email address is jose.galdo@carleton.ca. The authors would like to thank the comments from the editor and two anonymous referees, Brian Feld, Natalia Cantet, participants at the LACEA 25th meeting, the 3rd IZA workshop in skills and preferences in post-transition and emerging economies (St. Petersburg, Russia), and the DECON seminar series at the Universidad de la República, Uruguay. This work was carried out with financial and scientific support from the Partnership for Economic Policy (PEP) and with funding from the Department for International Development (DFID) of the United Kingdom (or UK Aid) and the Government of Canada through the International Development Research Centre (IDRC). The views and opinions expressed in this publication are those of the authors and do not necessarily reflect those of PEP. This study is registered at the AEA RCT with reference number AEARCTR-0002461.

1
# **1. Introduction**

Youth unemployment is a ubiquitous problem in most developed and developing countries. Over seventy-three million youth aged 15-24 are unemployed worldwide, and around 20 percent of the world’s youth are neither employed nor enrolled in an education or training program (International Labour Organization, 2017). For disadvantaged youth, who disproportionately lack skills and access to decent jobs that could lift them out of poverty, labor-market conditions are even more dramatic (International Labour Organization, 2015). In Mongolia, the setting of this study, the share of young individuals who were neither employed nor enrolled in an education or training program topped 25 percent (Shatz et al., 2015).

This study assesses the effectiveness of a vocational-training program on labor-market outcomes of disadvantaged youth in Ulaanbaatar, the capital city of Mongolia, following a standard randomized controlled trial design (hereafter, RCT). Vocational-training programs are the most widely used active-labor-market policy (hereafter, ALMP) intended to mitigate unemployment among poor youth worldwide, but no evidence exists regarding their effectiveness in central Asia. The most recent meta-analyses (e.g., McKenzie, 2017, Card et al., 2018, and Kluve et al., 2019) have revealed this important gap in the literature. Indeed, this large body of evidence shows modest (e.g., McKenzie, 2017) to positive (e.g., Kluve et al., 2019) effects of vocational-training programs on employment and earnings of youth in developing settings.<sup>2</sup> A solid pattern that has emerged from this literature is the substantial

> 2 McKenzie (2017) assessed the results of twelve RCT vocational training programs from eight different developing countries and reported an increase of 2.3 percentage points in employment and a 17 percent increase in earnings on average, although less than one-third of these twelve studies have reported statistically significant effects. Kluve et al. (2019) focused exclusively on vocational training programs that targeted youth following both experimental and non-experimental designs. Based on 113 studies from thirty-one developed and developing countries, they reported statistically significant impacts with effect sizes measured by 0.04 and 0.05

2
heterogeneity of estimated treatment effects as a result of variations in country settings, program design, and profile of targeted groups.

Mongolia offers a new setting for an assessment of ALMPs because it is a “transitional country”—that is, one whose labor market rapidly transformed from a centrally planned to a free-market economy. In fact, Mongolia’s economy shifted from a centralized, Communist system, in which youth unemployment was officially reported as very low or nonexistent, to a free-market economy with a high share of youth not in employment, education, or training (hereafter, NEET). Mongolia’s is a traditional herding and agricultural economy that transitioned into a resource-based economy in a short period of time. Two demographic shifts have added pressure to the labor markets over the past twenty years. First, the share of working-age youth is relatively high because Mongolia has a population with an average age of 26. Second, sustained internal migration from rural areas to the city of Ulaanbaatar has doubled the population of the capital city over that period. As a result, the rate of youth labor-force participation in Ulaanbaatar is remarkably low at around 40 percent. Thus, assessing whether vocational-skills training would foster employment and wage growth for young, vulnerable, and unemployed individuals in such an environment is relevant to policy.

The Mongolian Vocational Skills-Training Program (hereafter MVTP) follows a market-based approach in program design that is intended to match the skills that training institutions offer with those that are in demand in the market. This is sometimes known as a “demand-driven” ALMP. Such programs have become widely used in several Latin American countries, and assessment of their effectiveness shows positive effects on employment and earnings, particularly for disadvantaged women (Betcherman et al., 2007;

> SMD for employment and earnings, respectively. Effects sizes for developing countries are reported to be higher than those for developed countries.

3
Attanasio et al. 2011). “Demand-driven” training programs are designed to combine in-class training in selected vocational skills—where proven demand exists for those skills—with onthe-job training in the form of internships (see, e.g., Ibarrarán and Rosas, 2009; Galdo and Chong, 2012). In contrast to RCT training interventions implemented over the last decade in developing settings, a distinctive difference of the MVTP initiative is its short duration and low cost. The per-trainee cost ranges from USD $90 to USD $140 for trainings that vary from twenty to forty-five days.

Because lack of knowledge about market returns to training in a setting in transition from Communism to a market-based economy could lead participants to make uninformed choices, this study implements a complementary RCT design within a group of MVTP participants in which personalized weekly newsletters with information about the market returns to vocational training in Mongolia is distributed. The aim of the intervention was to shape choices made by MVTP participants regarding dropout or completion of vocational training. As shown Dammert and Galdo (2013) and Choe et al. (2015), lack of compliance with the treatment design is an important source of variation in the magnitude of treatment effects for training programs. Such a finding suggests the possible benefits of increased training exposure for potential dropouts. While providing information about market returns has been shown to be effective in education interventions in other developing settings (e.g., Jensen, 2010, Dinkelman and Martínez, 2013), to the best of our knowledge, the approach has not been tested as a policy design in the context of an ALMP in a developing country Main results from this studypoint out positive and meaningful effects on skills matching and self-employment. Estimated impacts for employment and earnings, on the other hand, are more ambiguous. The intent-to-treat estimated employment impacts are not statistically significant and range from -0.02 to 0.054, while the absolute change in monthly earnings ranges from USD $17 to USD $32, with statistical significance varying across 4
model specifications and time span. In line with what has been reported in the literature, these mean effects hide important heterogeneities across the sample of interest to the extent that positive and statistically significant effects on employment, earnings, and skill match are observed for relatively older, richer, and better-educated young people. Finally, provision of information on market returns to training positively affects the time of exposure to training (i.e., number of program days attended), but not lead to higher employment or greater earnings.

This article adds to the present understanding of youth (un)employment, particularly in transitional countries. Results are consistent with a broader body of micro-level evidence that suggests modest labor-market gains of vocational-training programs for youth. The study contributes to this evidence by assessing a low-cost, short-length training initiative, by providing some of the first evidence from central Asia, and by focusing attention on information constraints in returns to vocational training upon completion of training and employment. Indeed, these findings complement research regarding how providing information affects returns to formal education and school completion (e.g., Jensen, 2010). Although there are limits to generalizability and long-term effects, these results can provide insight into market-based approaches and information-based programs for ALMP in transitional settings **.** This paper is organized as follows: Section 2 briefly describes the Mongolian labor market, Section 3 provides details about the training design, Section 4 develops the evaluation experimental design, Section 5 discusses the data and baseline covariate distribution for treatment groups, Section 6 presents the results, and Section 7 provides some concluding remarks.

# **2. Institutional Setting**

5
Mongolia is a small, transitional country with a population of slightly above three million living in a landlocked area located between Russia and China. Ethnic Mongols account for about 95 percent of the population. Mongolia is one of the least densely populated countries in the world, with almost half its population living in the city of Ulaanbaatar, the country’s capital. Following the dismantling of the USSR in the early 1990s, Mongolia’s economy changed dramatically from a centrally planned system to a market-based one, and Soviet assistance, which had accounted for almost one-third of Mongolia’s GDP, disappeared almost overnight. Mongolia is resource-rich, with large deposits of copper, gold, coal, and uranium. Its mining sector accounts for almost one-fifth of GDP and 40 percent of exports, although it accounts for only 4.2 percent of employment. Mining has transformed Mongolia in a few decades from a traditional agricultural- and herding-based economy to a resource-based economy (World Bank Enterprise Survey, 2013). Today, Mongolia has a per capita GDP of USD $13,000 (PPP), which makes its economy comparable to that of South Africa or Sri Lanka. Agriculture nonetheless remains the largest employer in the country and still absorbs 30 percent of the labor force (down from 46 percent in 2003), followed by services (17 percent), public administration (15 percent), and manufacturing (10 percent) (World Bank Enterprise Survey, 2013).

Structural economic changes have been accompanied by two important demographic shifts that have imposed additional pressure on labor markets. Mongolia’s population is relatively young: the median age is 26 years, while the youth share of the working-age population is close to 55 percent. Likewise, the clearly declining trend of employment in herding and agricultural is mirrored by strong internal migration from provinces to the city of Ulaanbaatar. Indeed, since the late 1990s, population growth in Ulaanbaatar has averaged 5.8 percent, twice the country’s population growth rate. In 2013, 20 percent of the youth in Ulaanbaatar were new migrants who moved to the city in the past five years (Shatz et al., 6
# 2015).

Before the transition, the Mongolian labor market was characterized by high laborforce participation that reached more than 75 percent in the early 1990s. According to official statistics, unemployment was nonexistent because an “everyone should work” policy was in place. Only after 1992 did Mongolia begin measuring and reporting unemployment rates. Economic reforms implemented in the 1990s and afterward, including privatization and price liberalization, led to structural changes in Mongolia’s economy, vastly transforming Mongolian labor markets. Indeed, labor-force participation began to fall consistently while the youth unemployment rates increased dramatically. In 2013, the country-level labor-force participation rate was 56 percent and 42 percent, respectively, for men and women aged 1529. In Ulaanbaatar, the labor-participation rates of youth had fallen into the range of 30 percent for young women and 40 percent for young men (Shatz et al., 2015). Importantly, the share of inactive youth aged 15-29 has remained consistently above 20 percent. In 2013, the NEET rate for men and women aged 15-29 in Ulaanbaatar reached 13 and 27 percent. As in other developing settings, the Mongolian labor market shows stark gender disparities as a result of social norms and different employment frameworks for men and women (Pastore, 2010; Schmillen and Sandig, 2018). Social norms or deeply rooted expectations regarding the state’s responsibility for providing jobs might have played some role in this emerging picture. In the study sample, for instance, measuring the subjective belief that it is a “government responsibility to provide jobs to us” resulted in a mean response of 85 points (on a scale of 0 to 100 points).

On the demand side, the role of labor-market regulations that directly affect the costs of hiring and firing could constitute a relevant barrier to entry. Lehmann and Muravyev (2012) described the drastic transformation of labor-market legislation frameworks in transitional countries and their subsequent role in youth unemployment. According to the 7
World Bank Enterprise Survey (World Bank, 2013), less than 3 percent of Mongolian firms reported labor-market legislation as a constraint on labor demand. Yet, a quarter of sampled firms stated that the main obstacle to hiring was the inadequacy of skills in the workforce and mismatching between prospective employees’ skills and market needs. If such mismatching exists, it is important to assess the effectiveness of training programs tailored toward providing the vocational skills that are in demand in the market. One potential barrier for such investments could be the existence of compressed salary scales—typical under centrally planned economies—because firms might be unable to reward applicants who have additional skills. The evidence is, however, not clear in this regard. In 2012, large and increasing labor-market returns were observed for each additional year of schooling.<sup>3</sup> Nonetheless, firm-level data showed that only 22.6 percent of Mongolian enterprises had a performance-based wage system and only 7.9 percent had skill-based wage systems (Research Institute of Labor and Social Protection, 2017). Furthermore, although there is no legislation that prevents firms from adopting new screening technologies in the labor markets, the awareness and utilization rate of government employment services is very low (less than 20% of idle youth in Ulaanbaatar indicated they were aware of employment websites, electronic job boards, and other types of public support; see Shatz et al., 2015).

# **3. The Mongolian Vocational-Training Program (MVTP)**

The MVTP was introduced in 2003 to counteract high levels of unemployment and idleness. Its primary goal was to help those who were poor, unemployed, or vulnerable to unemployment find jobs through a combination of vocational-skills training and internships in business firms, and it remains the oldest and most extensive labor-market policy in effect

> 3 The annual rate of return for an additional year of schooling ranges from 7 to 10 percent (Batchuluun and Dalkhjav, 2019; Shatz et al. 2015).

8
in Mongolia. The MVTP follows key aspects of the so-called demand-driven training approach which is designed to align prospective employees’ skills with the real needs of local employers (Betcherman et al., 2007). The rationale for demand-driven training programs is twofold. First, private institutions can offer relevant, up-to-date training that decentralizes the traditional supply of vocational training by public institutions. Established training providers therefore usually bid to offer training in trades with high labor demand. Second, by combining traditional classroom education with on-the-job training in the form of internships, MVTP can provide training in trades that the market demands.<sup>4</sup> The Employment Promotion Service Center (hereafter, EPSC) and the Metropolitan Employment Department (hereafter, MED), both within the Mongolian Ministry of Labor, oversaw the implementation of this training program. The MVTP is financed by the State Employment Promotion Fund and, according to administrative data from the Ministry of Labor, the total number of participants was 8,000 by 2011 and total program expenditure was approximately 3.5 billion MNT (USD $2.1 million). The EPSC selects training institutions through a competitive bidding process. The selection criteria for training institutions include evidence of ability to provide adequate training, legal registration, curriculum quality, teaching quality, adequacy of training sites, and, importantly, the ability to place trainees in internship positions with registered private employers.

Although the MVTP began in 2003, its effectiveness has never been assessed. This paper focuses on the 2013 call that purposely used an RCT to identify and measure MVTP impacts. In that year, the EPSC selected forty-seven training institutions in Ulaanbaatar that offered 141 courses in six family vocational skills, including construction, hairdressing, cooking, and heavy-machinery operation. The length of training varied from twenty to forty-

> 4 The MVTP is different from “technical vocational training education,” which is considered a form of formal secondary education and last for years. This type of technical education has led to gains in employment and wages (Field et al, 2019).

9
five days, depending upon vocation, with a minimum duration of 144 hours per course. According to the program’s regulations, traditional classroom teaching did not exceed 30 percent of total hours, and practical, on-the-job training and internships accounted for the rest.

The MVTP tuition was set at between 140,000 MNT and 220,000 MNT per participant in 2013 (approximately USD $90-$140). These fees were paid directly by MED to training providers. The program offered no additional benefits such as transportation, meals, or insurance to trainees. As a result of budgetary constraints, the official number of training slots was set to 1,400 in 2013. Comparative information on course length, costs, take-up rates, attrition rates, and treatment effects across eleven youth training RCT interventions in developing countries is reported in Table 1. The length of training ranged from two-and-ahalf months to two years for classroom training plus two to six months for internships. The cost of training ranged from USD $330 to USD $1,700 dollars per trainee, with the exception of Uganda (Bandiera et al., 2020). Compared to other youth vocational-training programs, then, the MVTP was shorter and less expensive.

The contractual framework for the market based MVTP approach follows two levels. First, the EPSC signs contractual agreements with training providers regarding course length and content, timelines, and payments and internships for trainees. Second, individuals assigned to treatment are invited to sign an agreement with the corresponding labor office to secure participation. Up until Spring 2013, participants were required to sign a “trilateral” contract that involved a labor division district officer, a MED officer, and the beneficiary. This contract did not include any obligation on the trainees’ side to secure jobs after the completion of the classroom and internship phases. Subsequently, the MED changed to a “quadrilateral” agreement that added prospective employers as a fourth signatory. In practice, this meant that trainees needed to sign a job agreement with business firms to secure job 10
placements after the completion of the program. Although these job agreements were not legally binding, the institutional change created an important slowdown in registration and enrollment, which in turn affected the program’s take-up rates as shown in Section 5.

# **A Complementary Treatment Design**

Because no information exists regarding the effectiveness of this ALMP, or any similar intervention in Mongolia, the average participant might not be aware of the labor-market benefits of completing training. Indeed, administrative data showed that dropout rates were particularly high for the MVTP. This lack of information about the benefits of vocational training is even more important in the context of Mongolia’s very rapid transition from a state-controlled to a market-based economy. Less than 20 percent of unemployed Mongolian youth in Ulaanbaatar indicated they were aware of public-information sources regarding employment, while less than 5 percent made use of them in 2014 (Shatz et al., 2015).

Thus, inspired by the work of Jensen (2010) in the context of educational interventions, an information treatment is added to the study’s evaluation framework. Participants are randomly selected to receive weekly newsletters with information about labor-market outcomes among skilled and unskilled workers in Mongolia.<sup>5</sup> The newsletters stated market wages for occupations in sectors that were similar to those of the trainees and compared wages for jobs filled by unskilled workers. This information was estimated based on the 2012 Mongolian Household Survey. Figure S1.1 in the supplementary appendix available with this article at The World Bank Economic Review website shows a typical newsletter submitted to trainees.

> 5 In the context of formal schooling in the Dominican Republic, Jensen (2010) showed that students tended to underestimate the returns to formal schooling. When they were correctly informed, however, both enrollment in the subsequent year and the average length of formal schooling increased.

11
This intervention combines two salient features of a relatively new stream of literature in development economics that has highlighted the role of framing and message repetition (e.g., Chong, 2011). On one hand, it provides (new) information on market returns to vocational training to participants who are typically information constrained. On the other hand, it provides repetitive (weekly) information to target individuals with potential limited attention or present-bias individuals who, given a choice between a payoff today (dropout from training for work) and a pay off in the future (complete training for subsequent work), will choose to have the pay off now (Gabaix and Laibson 2006)”.

Information theory states that the value of information is determined by three important factors—credibility, novelty, and ability and willingness to act based on updated beliefs—all of which involve different forces and trade-offs (Hirshleifer and Riley, 1992). Individuals process new information largely based on their prior beliefs. If individuals change behavior in response to new information, then that information has value to them. Thus, the value of these newsletters among participants could not be considered a given. On one hand, disadvantaged participants do not have easy access to labor-market information, so it is arguably the case that information on training returns was new to the average participant. On the other hand, MVTP is a government program, and the erosion of trust in government on the part of disadvantaged populations might affect their perceptions of the credibility of information. Willingness to act based on new information could also be affected by commonly reported biases among unemployed individuals regarding their employment prospects (e.g., over-optimism). Such biases distort their behavior and may cause them to do little to leave unemployment (Spinnewijn, 2015). The ways in which such factors interacted each other, then, would determine the value of the information to participants.

# **4. Evaluation Design**

12
The randomization scheme for the two RCTs is covered in section 3 is depicted in the Appendix Figure S1.2. Prospective trainees aged 15-30 were assessed at their local district office. After completing a short baseline questionnaire, each applicant was screened for eligibility by an administrative officer. All suitable applicants were then sent to a district labor office in which each applicant, alongside a labor officer, chose her preferred trade courses on a first-come, first-served basis. Individuals who chose unavailable trade courses were not included in the experiment unless they opted for an alternative available trade.<sup>6</sup> At this stage, after prospective beneficiaries chose their courses, randomization at the individual level proceeded, and information from all district labor offices was pooled at the end of each day.

The random allocation of MVTP beneficiaries took place between August 26 and November 22, 2013. Each day during that period, the research team in coordination with MED officers, randomly assigned eligible applicants to either the treatment or control groups based on a 2:1 rule. As a result, the treatment group was made up of 774 trainees (65.2 percent), and 414 (34.8 percent) were assigned to the control group for a total of 1,188 participants.<sup>7</sup> Individuals randomly assigned to the treatment group were notified by telephone or through in-house visits about the program benefits, timelines, and regulations in the days that followed. Individuals who chose unavailable trade courses were not included in the sampling design.

Individuals in the treatment group were distributed across 141 courses at forty-seven training centers. Class size varied highly across training courses and ranged from three to thirty participants. The average number of MED-funded students was 9.6 per class. Actual

> 6 Although data on oversubscribed courses is not available, unavailable trade courses refer fundamentally to courses not being offered in the corresponding call rather than courses with high demand.

> 7 The sample size was originally set to 2100 individuals corresponding to 1400 in the treatment group and 700 in the control group to be able to detect a three percentage-point increase in employment with a power of 80 percent and a dropout rate of 30 percent. Unfortunately, and as a result of budget revisions, the Mongolian government slashed the number of potential beneficiaries for the 2013 call.

13
class sizes may have been slightly larger, however, because training institutions were entitled to recruit privately funded students if maximum class size was not reached. Though this happened occasionally, the significance was marginal, and most courses maintained small class sizes.

The second random allocation, i.e., the newsletter treatment, was implemented after beneficiaries were assigned to a vocational-training treatment group. The allocation of weekly newsletters was implemented at the class level, rather than at an individual level, to prevent spillover effects. 101 classes were randomly assigned to the treatment-newsletter group and forty classes to the control group. Thus, the final sample size was not known a priori and was determined after assigning classes to newsletter treatment and control groups.

The number of letters trainees received in the information-treatment group varied according to the length of training. Individuals assigned to the newsletter-treatment group received from 0 to 4 newsletters, with an average of 2.25. Across vocational trades, the number of letters varied from an average of 3.24 (cooking) to 1.66 (services). Because the total number of letters provided to participants varied depending on differences in the training, and because people who chose longer trainings may had unobservable characteristics that were correlated with training completion, the evaluation framework focused only on the comparison of the treatment and control information groups without considering the intensive margin of the treatment itself.

# **5. Data, Balancing Tests, and Take-Up Assessment**

The empirical framework used in this study is based on individual-level survey data, including a baseline, collected in Fall 2013 and two follow-up surveys administered six and eighteen months after training. The timeline of the intervention and data collection is 14
illustrated in Figure 1. The evaluation data includes sociodemographic variables, participation in formal schooling and training, labor-market outcomes, and detailed information on participation in the MVTP. Attrition reached 5.4 percent at baseline, 9.6 percent at the first follow-up survey, and 15 percent at the second follow-up survey. As shown in Table 1, and relative to similar interventions with similar timespan for the measurement of treatment effects, the attrition rate in this study is thus moderate.<sup>8</sup> Appendix Tables S1.1 and S1.2 provide results of tests for differences in baseline variables across treatment and control units for the six- and eighteen-month follow-up samples conditional on response status. For most baseline covariates, no systematic differences are observed between individuals in the treatment and control groups after conditioning on response status. Only for a small number of baseline covariates is the null of equality of means by treatment status rejected, after conditioning on response status. Finally, Appendix Table S1.3 reports the overall rate of attrition by treatment status. The null hypothesis that the overall attrition rate is of equal magnitude between treatment and control groups cannot be rejected.

Mean covariate balancing tests for the two experimental designs are shown in Table 2, ne for the allocation of training slots (left panel) and one for the allocation of weekly newsletters to individuals who received treatment (right panel). Baseline data show that the typical applicant is twenty-three years old, a woman (65 percent), and poor (82 percent lived in gers.<sup>_9_</sup> Almost half of the participants are married (45 percent), have children (42 percent), and live with their parents (47 percent). On average, they show high levels of formal

> 8 Table 1, Column 9, shows attrition rates that range from 18.5 (Card et al., 2011) to 46 percent (Cho et al. 2013). Alzua et al. (2016) reported no attrition because their reliance on administrative data.

> 9 Gers are traditional Mongolian dwellings. They are portable, round tents covered with skins or felt that are used by several distinct nomadic groups in Central Asia. Mongolia is divided into Ulaanbaatar, the capital, and twenty-one provinces (aimags). Ulaanbaatar is divided into six main and three small suburban districts (khoroos), which are further divided into smaller administrative units called khesegs. Poor khesegs are called gers, or impoverished neighborhoods, which, in most cases, lack water, drainage, and electric infrastructure.

15
schooling (51 percent had completed high school) and prior labor experience (60 percent). The p-values for the coefficients of OLS models that regressed treatment status on baseline covariates (left panel) are above 0.10 in all cases, indicating that individuals in the treatment and control groups came from the same population. These baseline characteristics are broadly similar to those of poor individuals aged 15-30 in Ulaanbaatar as reported in the 2014 Mongolian socioeconomic survey, which shows similar share of idle youth, age mean composition and average years schooling. The MVTP sample, however, shows a higher proportion of women (65 vs. 55 percent), a higher share of individuals living in gers (82 vs. 56 percent), smaller household size (4 vs. 6), and higher marriage rates (45 percent vs. 35 percent). Put differently, these differences suggest that the MVTP served, on average, a poorer set of Mongolian youth.

For the newsletter-treatment group, on the other hand, the right panel in Table 2 shows that the p-values for most variables do not reject the equality of means between experimentally determined treatment groups. Equality is rejected regarding a few variables, with the exception of those related to chosen vocations. This is expected because the random allocation of MVTP participants to the newsletter-treatment group occurred at the course rather than at the individual level.

Compared to other vocational-training programs for youth in developing countries (see Table 1), administrative data reveal that lack of full compliance with the treatment is relatively high for the MVTP. Out of the 766 applicants randomly assigned to receive vocational-skills training, 439 (57 percent) did show up for training, while other studies have reported take-up rates from 77 to 99 percent. Appendix Table S1.4 gives detailed information on enrollment numbers.

Self-reported survey information indicated that, among those who did not take-up the treatment, 35 percent cited family and personal commitments (e.g., household chores or 16
pregnancy), 30 percent had started a new job right after enrollment, and 31 percent said they were not able to comply with contractual MVTP requirements (i.e., quadrilateral contracts). From a policy standpoint, then, empirical assessment of the determinants of take-up becomes essential to gain insight both into the operation of the program (e.g., targeting, eligibility rules, and institutional requirements) and the identification and estimation of the parameters of interest.

The results from linear-probability models are shown in Table 3, where the dependent variable takes the value of 1 for those treatment-group individuals who received the treatment and 0 for those who did not. After including a rich set of sociodemographic and labor-market variables, MVTP institutional variables, and self-reported expectations regarding training and performance in the labor markets, results indicate that a handful of sociodemographic variables are statistically correlated with take-up decisions. On average, gender, household wealth, age, and formal schooling are important to take-up rates because women and individuals who were wealthier, older, and better-educated were more likely to participate in the program relative to men, poorer, younger, and less well-educated individuals. Importantly, findings show no meaningful statistical relationship between take-up and labormarket variables at baseline. This pattern ran counter to observations in other labor-market programs in which variables related to labor markets emerged as the main determinants of take-up.

Moreover, institutional variables related to the operation of the MVTP emerge as important take-up predictors. Individuals who were required to present quadrilateral contracts, for example, are fifty-four percentage points less likely to take-up the treatment relative to individuals asked to present trilateral contracts. This institutional requirement constituted a critical barrier for take-up. Likewise, the likelihood of attending the training is statistically related to the vocational-skills courses that participants chose. Individuals who 17
<!-- Start of picture text --> ¥, =a@t+BZ,+Xiy+t+t + €; €; (1)<br><!-- End of picture text --> ¥, =a@t+BZ,+Xiy+t+t + €; €; (1) ¥, i Z, <!-- Start of picture text --> Xj<br><!-- End of picture text --> Xj Ej
lasted, Equation 1 also includes fixed effects by day of random assignment, τi.<sup>10</sup> The effective treatment-on-the-treated (TOT) parameter, on the other hand, is estimated by the 2SLS estimator following an instrumental-variable approach in which actual participation in the training program (T) is instrumented by treatment status (Z): Appendix Table S1.5 shows the first-stage estimation results. The coefficient associated with the instrumental variable Z is statistically significant at the 1% level, and the resulting F- statistics confirm the relevance and strength of the instrument.

The main results for the vocational training intervention are presented in Table 4. Four outcomes of interest are examined: employment in the week prior to the survey, monthly earnings, skills matching, and self-employment. Skills matching is defined by a dummy variable that takes the value of 1 if an individual self-reported using trained vocational skills at current work, 0 otherwise.<sup>11</sup> Appendix Table S1.6 provides a full description of all outcome variables. The upper panel shows short-term (six-month) treatment effects, while the lower panel shows medium-term (eighteen-month) mean effects. Odd- and even-numbered columns show results for specifications without and with control variables. Robust standard errors are reported in parentheses.

Short-term point estimates (six months) show positive and statistically significant effects for self-employment and skills matching regardless of the econometric specification

> 10 Estimations were carried out for three different specifications of the outcome model. First, a specification with no controls. Second, a specification in which control covariates selected by post-double-selection LASSO were added (Belloni et al., 2014). After considering a large set of baseline variables (+35), only four control variables were selected: gender, whether the respondent had ever worked, whether the respondent signed a “quadrilateral” contract, and whether the respondent resided in Nalaik district. A third specification added standard variables highlighted in human capital models of wage regressions (age, gender, and schooling) to the selected LASSO controls. The main table of results reports only the first and second specification because the last one yield quantitative results similar to those of the second specification.

> 11 This included any previous training that the respondent had completed in any public or private training institution, including technical vocational education and training (TVET) education.

19
and parameter of interest. Relative to the mean of the control group, the magnitude of the treatment effects is somewhat large, particularly for the TOT point estimates (from 0.099 to 0.139 for skills matching and from 0.083 to 0.085 for self-employment). These results suggest that the vocational skills of MVTP participants are better aligned to occupations as compared to individuals in the control group. Similarly, and although the incidence of selfemployment remained very low in comparison to what is commonly observed in other developing countries, these point estimates indicate that self-employment more than doubled among MVTP participants relative to non-participants. For the employment outcome, on the other hand, positive but imprecisely measured treatment effects are observed, ranging from 0.037 to 0.054 (ITT) and from 0.076 to 0.127 (TOT).<sup>12</sup> Finally, earnings effects are positive across all specifications, but their statistical relevance depend upon both the use of control variables and alternative specifications for the outcome itself. In this regard, significant and sizable impacts arise whenever LASSO control variables are included in the specification models, while imprecisely measured earnings effects emerge when the econometric specifications do not incorporate control variables (see Table 4). The results of sensitivity tests that use the inverse hyperbolic sine transformation, trimming, and winsoring for earnings outcome are shown in Appendix Table S1.7.<sup>13</sup> While statistically significant results hold under trimming and winsoring approaches, imprecise point estimates emerge when the distribution of the outcome is smoothed out. All in all, the magnitude of the mean effects for monthly earnings is somewhat large (as high as 23% for the ITT) from a mean of the control group of approximately USD $110.<sup>14</sup>

> 12 In terms of percent gain on the control employment rates, the 95 percent confidence interval for employment ITT effects (column 2 in Table 4) is [-3, 27] and [-11, 15] six- and eighteen-months following treatment, respectively.

> 13 Winsorization and trimming used the top 95 percentile of earnings as the cut-off value.

> 14 In terms of percent gain on the control income, the 95 percent confidence interval for earnings ITT effects (column 4 in Table 4) is [-2, 45] and [-1, 42] six- and eighteen-months following treatment.

20
Mean effects eighteen months after completion of training are presented Table 4. Relative to short-term effects, a decay in the magnitude and statistical significance of treatment effects emerges for all outcome variables except self-employment. The effects on skills matching dissipate and become statistically not significant, while effects for the employment outcome become either negligible or negative. For monthly earnings, the point estimates from both ITT and TOT parameters are still in line with short-term findings because smaller but positive statistically significant effects emerge depending on the specification used. Self-employment, on the other hand, is the only outcome that shows steady positive and statistically relevant impacts over time with sizable point estimates ranging from 0.04 to 0.092.

By comparing the MVTP confidence intervals for earnings ITT treatment effects, which are measured in terms of percent gain relative to the control group and reported in footnote 13, against the corresponding findings that emerged from eleven RCT vocationaltraining programs that targeted youth worldwide (see Table 1), somewhat sizable effects can be observed. This is particularly interesting given the short duration and low cost of this training initiative. As a result, and although there is uncertainty associated with the gains in earnings, a straightforward cost-benefit calculation reveals that it requires 6 to 8 months of treatment effects to recoup the costs of the courses. This is plausible given the short-term impacts depicted in Table 4.<sup>15</sup> These findings are also in line with other studies that measured effects over a longer time span and found that treatment effects fell further over time. (e.g., Alzúa et al., 2016; Ibarrarán et al., 2019). Overall, results suggest that training helped trainees move to tailored

> 15 This calculation considers the explicit costs of training that range from USD $90 to USD $140 plus implicit (forgone) costs that are computed as one month of control group mean earnings times the probability of being employed. Monthly benefits are computed using the statistically significant ITT monthly earnings depicted in column 4 in Table 4 (Tughrik $53255, or equivalent to approximately USD $27). A conservative skills depreciation rate (20 percent per year) is assumed for this calculation.

21
jobs right after treatment ended but, over time, the MVTP fell somewhat short of achieving its promises. This is plausibly explained by the short duration of employment that is likely secured at business firms that offered internships. However, extending these job contracts could have not been a priority for many business firms over time. Unlike most developing countries, employment in Mongolia is mostly formal, and thus, extending job contracts to disadvantaged youth that participated in short vocational training courses might not be a priority for many business firms. This, in turn, might explain the consistency of positive impacts for self-employment.<sup>16</sup>

# **Heterogeneous Effects**

A solid pattern in the existing training literature is the substantial heterogeneity in the impacts of training **.** To account for the heterogeneity of effects across subgroups of participants, the estimation framework and model specification given in Equations 1 and 2 are applied after interacting the treatment status variable with baseline covariates of interest: gender (men vs. women), age (15-21 vs. 22-30), poverty status (poorest vs. less poor), and educational attainment (less than high school vs. high school vs. technical and university education).<sup>17</sup> These policy variables, commonly used in the assessment of vocational-training programs worldwide, are related to the efficiency of the targeting approach. As before, the same four outcomes of interest are monitored, at six (Table 5A) and eighteen months (Table 5B) after the intervention.

The results highlight the heterogeneity of vocational-training effects across

> 16 In Ulaanbaatar, informal employment for the youth is estimated to be lower than 15 percent, way below the observed informal employment rate in Latin America or African countries. 17 An issue to consider when comparing the ITT and TOT impacts is that take-up rates vary with some of these demographic variables. Thus, while ITT estimates captures the combined effect of any differences in take-up across groups with differences in the effectiveness of training by group, TOT estimates are subject to the concern that complier status differs across demographic groups.

22
demographic groups. Individuals aged 15-21, normally the demographic group at the highest risk of unemployment, benefit least from the program six months after treatment (see Table 5A). According to TOT point estimates at six months after training, the likelihood of employment, self-employment, and skills matching is, respectively, 29, 19, and 5 percentage points lower for the youngest group relative to the 22-30-year-old cohort. Eighteen months after the intervention, however, these differences are no longer statistically significant. This suggests a rapid deterioration of MVTP effects over time for all age groups rather than an improvement in labor-market outcomes for the youngest vs. the older cohorts. Consistent (negative) differential effects for monthly earnings also emerged for the youngest group six and eighteen months after the intervention. Interestingly, men and women benefit equally across all outcomes of interest, as shown in the second panel of Tables 5A and 5B. This important result is contrary to what has been observed in similar demand-driven approaches in other countries, particularly in Latin America, as shown by Attanasio et al. (2011) for Colombia; Card et al. (2011) for Dominican Republic; Alzúa et al. (2016) for Argentina; and Díaz and Rosas (2016) for Perú. All these studies showed that young women benefited more from these types of vocational training initiatives than did young men. Nonetheless, the point estimates for the independent gender variable indicates that men have much higher employment rates and earnings than do women, reflecting reported large gender gaps in Ulaanbaatar’s labor markets (Pastore, 2010; Shatz et al., 2015).

While the MVTP was originally designed to target youth from poor households, findings show that people in the bottom quartile of the household asset index benefit less from the program (see Tables 5A and 5B). In comparing those at the bottom of the poverty index to those at the middle and upper end of the distribution, the ITT and TOT parameters show statistically significant differential coefficients for employment (minus sixteen and minus thirty-nine percentage points) and self-employment (minus seven and minus sixteen 23
percentage points). These sizeable differences in treatment effects hold eighteen months later, although statistical significance is only observed for the self-employment outcome.

Monthly earnings for the poorest among the poor, on the other hand, show negative differential effects at six and eighteen months after the intervention. Statistical significance, however, is uncertain as it depends upon the specific definition of the earnings outcome (levels vs. inverse sin transformation) and time span (six vs. eighteen months).

Finally, sizable heterogeneous effects emerge based on participants’ formal level of schooling. Individuals with less than a high-school education and those who had completed high school show negative differential effects in comparison to participants with technical or university education. These differences are monotonic with respect to education level. In particular, the magnitude of these differential effects is striking for those at the bottom of the schooling ladder and for all outcomes of interest. The TOT point estimates at six and eighteen months after the intervention show, for instance, that the likelihood of employment is fiftynine and seventy-seven percentage points lower for individuals with the least amount of formal schooling in comparison to those at the upper end of the schooling distribution. These substantial differential effects hold eighteen months after the intervention.

Importantly, all sub-groups for each variable of interest contain at least 20% of the sample, and these sizable heterogeneous effects are therefore not driven by small sample sizes of sub-groups. Neither is the assessment fundamentally affected when correcting for multiple-hypothesis testing by the Hochberg’s false-discovery-rate method.<sup>18</sup> Rather, these heterogeneous results reflect the contextual reality of Ulaanbaatar, where substantial gaps in

> 18The Hochberg’s test, which is implemented at level δ=0.10, controls the proportion of false positives within the set of rejected hypotheses. After implementing this test for the heterogenous results, it still rejects the null hypothesis for the point estimates that are reported in Tables 5A and 5B, holding (unadjusted) statistical significance at the 1% or 5% level. However, this test does not reject the null hypothesis for the point estimates that are reported in Tables 5A and 5B holding, (unadjusted) statistical significance at the 10% level. In this regard, correcting for multiple-hypothesis testing leads to the loss of statistical significance for the differential effects of both age on skills matching and of poverty status on self-employment six months following the intervention. For the medium-term effects, statistical significance is lost for the differential impacts of poverty status on earnings.

24
labor-market outcomes across age, education, and poverty ranks have been similarly observed for the youth population. The 2012 Mongolian Labor Survey shows, after controlling for a rich set of observable covariates, that young individuals with university education earned on average 50 to 60 percent more than individuals with secondary education, while individuals with a primary education earned between 18 and 23 percent less than people with secondary education. Likewise, individuals aged 25-29 reported 53 percent more earnings than individuals aged 15-19, while the NEET rates by poverty status ranged from 40 percent (poor) to 11 percent (non-poor) (Shatz et al. 2015). It would have been extremely difficult for a short-term program like ours to change these structural labor-market gaps.

Furthermore, because bias toward optimism has been reported to be particularly striking among the unemployed in other settings (e.g., Spinnewijn, 2015), treatment-effect heterogeneity by subjective beliefs regarding job prospects is also assessed. Measuring at baseline a subjective belief in finding a job within the successive six months, a median of eighty points is recorded in the sample (on a scale of 0 to 100), while around 70 percent of the sample self-reported being “very optimistic” about getting a job when answering a categorical-type similar baseline question. Compared to the actual number of employed individuals (46 percent) six months following the baseline, the sample of youth unemployed seems to show indeed over-optimistic beliefs regarding their job prospects.

Point estimates for the vocational treatment variable interacted with the dummy variable “very optimistic beliefs” is depicted in the bottom of Tables 5A and 5B.In the shortrun, positive differential effects emerge for individuals who are optimistic about their labor market prospects relative to individuals with lower expectations. These differential effects are statistically significant for employment and earnings. In the medium-term, positive 25
differential effects are still observed for those with optimistic beliefs regarding their labor prospects, but statistical precision is lost.

All in all, these results suggest that subjective beliefs regarding job prospects may hinder the effectiveness of vocational training among unemployed youth who hold pessimistic beliefs about their job prospects. However, this evidence should be taken with caution and only as suggestive because subjective beliefs may have been correlated with unobserved factors that are themselves associated with the outcomes of interest.

The effect of treatment on those who would have attained (un)favorable outcomes in the absence of training is also assessed. Because random assignment does not identify the joint distribution of potential outcomes with and without training, the endogenousstratification method proposed in Abadie et al. (2018) is used. Separate analyses are conducted for three subgroups of participants, defined in terms of intervals of the predicted outcome without training. Two alternative sets of baseline variables are used to estimate an index that reflects each participant’s predicted outcome in the absence of treatment and implement the leave-one-out estimator to minimize bias in the estimation of the ITT parameter as a result of the use of in-sample information on the outcome variable to stratify the sample.<sup>19</sup> ITT parameters are displayed in Table 6, together with bootstrap standard errors for the adjusted endogenous stratification estimator. Results show heterogeneous intent-to-treat gains across three predicted-outcome groups. Regardless of the outcome of interest and time length for impact analysis, large and positive impacts emerge for the high-predicted outcome group, while negative or close to zero effects are observed for individuals with low-predicted

> 19 In the first step, the outcome is predicted in the absence of the treatment using two sets of information to assess the sensitivity of results to the selection of baseline variables: a basic set of socio-demographic variables and a rich set of +35 baseline variables. In the second step, an adjusted-outcome specification is implemented by using the same specification employed in the computation of the main treatment effects—i.e., fixed effects by day of random assignment and LASSO selected control covariates.

26
outcomes. Statistical significance, however, varies according to the outcome of interest, set of information used in the prediction of outcomes, and time span. The clearest pattern emerges six months following treatment as one observes positive and statistically significant effects for the high predicted outcome group. These estimates suggest that the MVTP improved labor-market outcomes largely for individuals who would have obtained favorable outcomes in the absence of treatment.

# **Effect by Field of Training**

Knowing whether mean effects vary according to the field of study could be important for policy because it may signal the importance of providing participants with training in certain fields rather than in others. Intent-to-treat point estimates according to field of study are presented in Appendix Table S1.8. Rather than considering only one treatment indicator, multiple treatment variables in Equation 1 were incorporated, according to the field of study. Each of these treatment indicators takes the value of 1 if it refers to the specific “X” field of study and is 0 otherwise. The following categories were included: mechanical/machinery, hairdressing and beauty services, craftsmanship, agriculture and gardening, cooking and baking, and a dummy for all other vocations, which together accounted for all training courses. These point estimates should be taken as merely indicative and assessed with caution, however, because sorting or self-selection of trainees into specific fields of study may have been driven by unobserved factors (e.g., personal traits) which are, in turn, correlated with the outcomes of interest.

Results indicate some impact heterogeneity for the MVTP according to the chosen field of study in the short run, although most of the parameters lack statistical significance. “Mechanical/machinery” and “hairdressing and beauty services” show positive and significant effects, with the latter showing the largest effects across all outcomes of interest six months after the intervention. These heterogeneous differences by field of study tend to 27
dissipate across all fields of study at eighteen months following treatment, however. Still, “hairdressing and beauty services” is the only field of study that shows positive and significant mean gains for the self-employment outcome eighteen months after the intervention, while “agriculture and gardening” and “craftsmanship” show negative differential effects for earnings, skills match, and self-employment outcomes. This is consistent with the sectoral earnings distribution in Mongolia, which shows that average earnings in agriculture and related occupations are on average 50 percent lower than in services.

# **Newsletter Treatment**

Out of the 410 trainees who attended 141 training courses, 291 were assigned to the newsletter-treatment group (101 courses) and 119 to the control group (forty courses). Not all 291 individuals received the newsletters because some of them never showed up at training centers. However, the focus is placed on the estimation of ITT treatment effects without any formal distinction between ITT and TOT parameters because the rate of no-shows was relatively low for the newsletter treatment. Four intermediate outcomes of interest are assessed: days the respondent attended MVTP, whether the respondent received MVTP qualification (passed the exam), whether the respondent received an MVTP certificate (formal graduation from the program), and respondents’ dropout rates. All outcome variables were self-reported with the exception of the last because administrative information was not recorded in a systematic way.

The results show positive (3.7) and statistically significant impacts for “days attended training”, as depicted in the upper panel of Table 7. Clustered standard errors by class level are shown in parentheses and, similar to the estimation of MVTP main effects, a post-double28
selection LASSO estimator is implemented for control variables.<sup>20</sup> Relative to the average days attended by trainees in the newsletter control group (twenty-two), this represents an increase of 17 percent. Consistently, the average effects on MVTP qualification (the respondent passed the exam) are positive (0.045) but lack statistical precision. This suggests that trainees updated their behavior in response to new information on market returns to training and thus, that the new information might have had some value to them. This finding is in line with the relatively recent stream of literature in development economics that has successfully used information policies to improve economic outcomes in developing settings (e.g., Chong, 2011). For the other two intermediate outcomes, certification and dropout rates, unexpected signs and uninformative treatment effects are observed. The dropout rate is rather low and the sample size provides too small a level of power for detection of meaningful effects to occur.

Heterogeneous effects for the newsletter intervention are reported in the lower panel of Table 7Unlike the impact of vocational training, which was of benefit to relatively betteroff young participants, the newsletter-related intervention shows no statistical differences in average gains across demographic groups or over time. Indeed, no statistically significant heterogeneous effects by age, gender, poverty, or level of schooling arise. The lack of statistical significance is likely related to small sample sizes across all subgroups.

A related policy question is whether the combination of vocational training with newsletters about returns to training leads to higher earnings and employment for MVTP participants. Because both interventions are orthogonal to each other, two independent variables are constructed, one that takes the value 1 for individuals who were offered both treatments, 0 otherwise, and a second dummy variable that takes the value of 1 for

> 20 For the administrative dropout outcome, the LASSO approach selected two control variables: whether responded was a student at baseline and whether respondent was enrolled in technical vocational education at baseline. No control variable was selected for the other outcomes of interest.

29
individuals who were offered only the vocational training, 0 otherwise. Point estimates and p-values that test the null of equality between these independent variables are shown in Table 8. The estimation model follows the same specification used in the computation of the main treatment effects given in Table 4. While the magnitude of the coefficient associated with both treatments is higher than that for vocational training alone across most outcomes of interest, no statistically meaningful differential results are observed at six and eighteen months after the intervention. The p-values shown at the bottom of Table 8 do not reject the null of equal coefficients across all four outcome variables of interest. This result is arguably related to the short length of training offered by the MVTP, since attending three extra days of training might be marginal to change the employment and earnings odds of those receiving the newsletter treatment.

# **7. Conclusions**

This study examined the effectiveness of the demand-driven Mongolian Vocational Training Program in Ulaanbaatar, the capital of Mongolia. The setting of this evaluation is new to the literature because little is known about Mongolia’s labor markets considering the country’s relatively recent transition from a centrally planned economy, wherein unemployment was set to zero by law, to a market economy. Like other demand-driven vocational-training programs implemented since the 1990s, the MVTP aimed to counteract high levels of youth unemployment and idleness by responding to actual labor market needs through a mix of traditional classroom courses and on-the-job training (internships). This study implemented a complementary random allocation design to assess the role of information about market returns to training on the length of exposure to training.

One striking result is the low take-up rate for the intervention. Around 42 percent of individuals randomly assigned to training did not attend the courses. Analysis of the 30
determinants of take-up showed that institutional constraints, notably the requirement that quadrilateral contracts be signed prior to the start of the course, as well as the chosen field of study and some demographic variables (gender, education), played an important role in takeup decisions. In particular, the quadrilateral-contract requirement, which seeks to secure jobs for trainees after the completion of the program, seemed to be the major barrier to participation, and its role cannot be understated. Scaling-up this vocational training initiative would be problematic under the current contractual framework.

Overall, positive average effects of the training intervention were observed on outcomes of interest. However, statistical significance varies across variables, model specification, and over time. Informative results are mainly observed in the short run, while a distinctive decay in the magnitude of effects is observed eighteen months following the intervention. Given the short duration and thus low cost of this MVTP initiative, the magnitude of the treatment effects are somewhat sizable, and thus can inform policy about the relative success of implementing market-based ALMP approaches in settings where, prior to an economic transition, the state had been the sole provider of worker training, and individuals had little choice regarding the training to which they were assigned.

As in the case of other ALMPs implemented worldwide, results showed that not everyone benefited equally from the program, which highlights the importance of policies that target both participants and the design of training content. In fact, heterogeneous effects emerged because relatively better-off, older, and better-educated students benefitted disproportionately from the intervention. Such results indicate that the MVTP should adjust its targeting design if the idea is to help those most in need.

Finally, this study found that providing information to young participants on market returns to training had positive, statistically significant effects on the length of exposure to training (days trainees attended the program). Given the low cost of providing newsletters to 31
young participants who are arguably information-constrained, such an approach could become a standard component of vocational-training programs or the scope of such programs could be expanded to include information on returns to specific trades before prospective trainees make choices.

32