Recipient views of program impact PAIDEC CONACyT CONACyT 2000

*IDB—Inter-American Development Bank.

# _Early Program Evaluations_

All four evaluations of CIMO-PAC (1995, 1997, 2002, 2005) and the CONOCER evaluation can be characterized as non-experimental by their use of an ex-post matching to form comparison groups in order to isolate the causal impact of program participation. In all cases except CIMO-PAC’s 2002 evaluation, random sampling was unnecessary because the universe of firms or workers meeting evaluation criteria were so few.

In the 1995 and 1997 evaluations of CIMO, the control group was made up of 316 firms from the economic census that shared size, sector and location characteristics with those of the pilot treatment group. The treatment group consisted of all firms that entered the program in 1991 and 1992 and remained registered in 1993. Though 442 firms met these criteria, only 248 contained the necessary information for evaluation. This may have led to selection bias, because only the best-managed firms who kept organized records were used in the evaluations. This in turn could have led to overestimates of the effectiveness of the program.

Similarly, the control group used in the CONOCER evaluation was selected from the same sectors of activity and region as the treatment group, with the intention of matching firm characteristics. The treatment group consisted of firms of those sectors and regions in which the greatest number of labor competency certificates had been issued. The group was selected as such because in 2001 only a few sectors had completed the cycle of supports provided by CONOCER.

In the CIMO-PAC 1995 evaluation, a series of regressions were run on productivity, employment and wages using dummy variables to isolate the effect of the program by sector, type of support and firm size. The CIMO-PAC 1997 analysis incorporated regression based on a Cobb-Douglas production function. In the case of CIMO-PAC 2002 evaluation, the analysis consisted of comparing the percentages of responses of the survey questions between the treatment and control groups. Cost benefit studies were also carried out in the CIMO-PAC evaluation (1995 and 1997). In each case the treatment and control groups were compared based on the unit cost per unit impact. Cost benefit indices were calculated based on impact measures divided by private and public costs associated with the program.

The evaluations produced mixed results in relation to program outcome and impact objectives. Some programs seemed to have more impact than others, and notable differences were found in the impact of firms of different sizes and in different sectors. For instance, the CIMO-PAC (1995) evaluation concluded that the treatment group’s employment increased 8.5 percent while the control decreased 1 percent between 1991 and 1993. The CIMO-PAC (2002) evaluation concluded that while 16.5 percent of the treatment group’s sales were exported, the control group firms only exported 9.7 percent of their sales over the same period. The CONOCER evaluation found large differences in outcome indicators between certified and non-certified workers in the forestry and textile industries, but the differences 14