Using administrative data from EF, we examine the rate of program take-up by the treatment and control groups for the 2010, 2011 and 2012 cohorts in Table 6.<sup>44</sup> The table shows a high degree of uptake (65 to 74 percent) among the treatment group, but also a high rate of participation among the control group.<sup>45</sup> Between 26 percent and 36 percent of the individuals in the control group participated in the EF training course that they applied for, even though their scores did not qualify them for admission.<sup>46,47</sup> In the presence of imperfect compliance, standard impact evaluation methods produce intent-to-treat (ITT) estimates, as described in section IV. However, the relatively high degree of noncompliance in our study leads to a likely downward bias in our ITT estimates, which compare a treatment group in which not everyone received treatment to a control group in which some people did receive treatment, hence diluting the impact of the program. We present an alternative set of ATT estimates in Annex 3 (available upon request from the authors) in which we compare those who actually complete training to those who do not, irrespective of treatment assignment.<sup>48</sup> In our view, the ITT results serve as a lower bound for the estimated program impact while the ATT results serve as an upper bound.

# **VI. Short-Run Results of the EF Program**

This section describes the impact of the Employment Fund’s training programs for the combined 2010, 2011 and 2012 samples. We measure outcomes approximately one year after the start of training.<sup>49</sup> We start with a description of our analysis on likelihood of being in the treatment group. We proceed with the shortterm impacts of the EF program on the full sample and on various demographic and socioeconomic subgroups.

## **a. First-Stage Estimation of Propensity Scores**

At the heart of the propensity score matching method is pairing trainees with non-trainees who are similar in terms of their observable characteristics. This pairing between a trainee and comparable non-trainees is done via a propensity score, as described in section IV.

> 44 An individual is recorded in the EF monitoring database as a trainee only when the T&E provider submits the person’s name at the end of the training. Since the EF only reimburses T&E providers at the conclusion of the training, they do not record nor do we have any way to track how many people enrolled, but did not complete, the training in 2010 to 2012.

> 45 The monitoring records indicate that the control group individuals who received training did so in the original training course that they applied for. There is no evidence that they applied for and participated in a different EF training course sponsored by a different T&E provider, nor that they participated in EF training courses in a different year.

> 46 In practice, while we cannot be sure what happened in each case, the T&E providers probably dipped into the pool of lowerscoring applicants in an attempt to fill up the training slots as people dropped out.

> 47 The stable unit treatment value assumption (i.e., SUTVA) based on Rubin (1980) assumes that (1) the treatment status of any unit does not affect the potential outcomes of the other units (i.e., non-interference) and (2) the treatments for all units are comparable (i.e., no variation in treatment). Note that take-up of training among control group units is not a violation of the SUTVA, as take-up among non-beneficiaries was not directly a result of the take-up of training among the individuals selected for training.

> 48 The ATT estimates suffer from an additional source of bias, since it is likely the “best” members of the control group who gained access to the program and are reassigned to treatment in the ATT analysis, and the least motivated among those assigned to treatment who fail to participate and are reassigned to control. This is discussed further in Annex 3, available upon request.

> 49 Because the EF-sponsored training courses vary in length from 1 to 3 months, the follow-up survey examines outcomes 9 to 11 months after the end of the training.

15