metadata: authors: 0: author: Hong Tan
count: Chile
docna: 0: docna: Evaluating SME support Programs in Chile
            using panel firm data
docty: Policy Research Working Paper
subtopic: Investment and Investment Climate,Construction Industry,Common Carriers Industry,Food & Beverage Industry,General Manufacturing,Pulp & Paper Industry,Plastics & Rubber Industry,Labor Markets,Export Competitiveness,Business Cycles and Stabilization Policies
teratopic: Information and Communication Technologies,Social Protections and Labor,Poverty Reduction,Finance and Financial Sector Development
abstracts: cdata!: This paper evaluates small and medium
            enterprise (SME) support programs in Chile using a
            firm-level panel for the 1992-2006 period on two groups of
            firms  a treatment group that participated in SME programs
            and a control group that did not. These unique panel data
            provide an unprecedented opportunity to address several
            issues that have plagued impact evaluations of SME programs
             selectivity bias from observed and unobserved firm
            heterogeneity, identification of an appropriate control
            group, and inability to track firms over a long enough
            period of time for performance outcomes to be realized.
            Using difference-in-differences models combined with
            propensity score matching methods, the paper finds evidence
            that participation in SME programs in Chile is associated
            with improvements in intermediate outcomes (training,
            adoption of new technology and organizational practices),
            and causally with positive and statistically significant
            impacts on sales, production, labor productivity, wages and
            exports. The mixed results of previous studies may be
            attributable in part to the confounding effects of
            unobserved heterogeneity motivating selection into programs
            of firms with relatively low productivity levels, and in
            part to time-effects of program participation occurring in
            years after the time horizon of most impact evaluation studies.
geo_reg_key: 119228,517191
repnme: repnme: Evaluating SME support Programs in Chile
            using panel firm data
display_title: Evaluating SME support Programs in
            Chile using panel firm data
listing_relative_url: /research/2009/10/11233004/evaluating-sme-support-programs-chile-using-panel-firm-data
topicv3: Innovation, Technology Transfer and
            Diffusion m1326019 1084,Investment Climate m1326293 405,Manufacturing Sectors m1326521 1281,Labor Productivity m1504060 2756,Export Promotion m1504271 2566
publishtoextweb_dt: 2010-07-01T00:00:00Z
historic_topic: Information and Communication Technologies,Social Protections and Labor,Poverty Reduction,Finance and Financial Sector Development
no_of_pages: 41
model_extractions: WPS5082 Policy Research Working Paper 5082 Impact Evaluation Series No. 39 Evaluating SME Support Programs in Chile Using Panel Firm Data _Hong Tan_ The World Bank Latin America and Caribbean Region Poverty Reduction Division October 2009
Policy Research Working Paper 5082

# **Abstract**

This paper evaluates small and medium enterprise (SME) support programs in Chile using a firm-level panel for the 1992-2006 period on two groups of firms—a treatment group that participated in SME programs and a control group that did not. These unique panel data provide an unprecedented opportunity to address several issues that have plagued impact evaluations of SME programs —selectivity bias from observed and unobserved firm heterogeneity, identification of an appropriate control group, and inability to track firms over a long enough period of time for performance outcomes to be realized. Using difference-in-differences models combined with propensity score matching methods, the paper finds evidence that participation in SME programs in Chile is associated with improvements in intermediate outcomes (training, adoption of new technology and organizational practices), and causally with positive and statistically significant impacts on sales, production, labor productivity, wages and exports. The mixed results of previous studies may be attributable in part to the confounding effects of unobserved heterogeneity motivating selection into programs of firms with relatively low productivity levels, and in part to timeeffects of program participation occurring in years after the time horizon of most impact evaluation studies.

This paper—a product of the Poverty Reduction Division, Latin America and Caribbean Region—is part of a larger effort in the region to evaluate the economic and social impacts of policy interventions. Policy Research Working Papers are also posted on the Web at http://econ.worldbank.org. The author may be contacted at htan@worldbank.org or hongwtan@ comcast.net.

_The Impact Evaluation Series has been established in recognition of the importance of impact evaluation studies for World Bank operations and for development in general. The series serves as a vehicle for the dissemination of findings of those studies. Papers in this series are part of the Bank’s Policy Research Working Paper Series. The papers carry the names of the authors and should be cited accordingly. The findings, interpretations, and conclusions expressed in this paper are entirely those of the authors. They do not necessarily represent the views of the International Bank for Reconstruction and Development/World Bank and its affiliated organizations, or those of the Executive Directors of the World Bank or the governments they represent._ Produced by the Research Support Team
# **Evaluating SME Support Programs in Chile Using Panel Firm Data**<sup>**1**</sup>

## Hong Tan

_Keywords:_ small and medium enterprises, program impact evaluation, Chile _JEL codes:_ L25, L53, O25, O54 _Contact information:_ <u>htan@worldbank.org</u> OR <u>hongwtan@comcast.net</u>

> 1 This country study was funded by a research grant RF- P105213-RESE-BB from the World Bank’s Research Committee for the project “Evaluating small and medium enterprise support programs in Latin America” managed by LCSPP and directed by Gladys Lopez-Acevedo and Hong Tan.  We acknowledge José Guilherme Reis for providing us with access to the Chile Investment Climate Survey and ENIA panel, and Michael Goldberg whose study provided invaluable background information on SME programs in Chile.  We thank David McKenzie in particular for his expert guidance on key methodological issues that arose over the course of the study.
# **Evaluating SME Support Programs in Chile Using Panel Firm Data**

## **I.  Introduction**

Many governments provide subsidized business development services (BDS) and finance to promote productivity improvements and exports, create jobs and improve competitiveness among their small and medium enterprises (SMEs). These interventions are often justified on the grounds that SMEs face diseconomies of scale, imperfect information about markets, production methods and new technology, and limited access to finance as compared to their larger counterparts. Questions arise not only about whether these assertions are warranted, but also about the effectiveness of the SME support programs designed to address these market imperfections and constraints. The questions are not surprising given the paucity of impact evaluations of such support programs, especially in developing countries. What little rigorous evidence exists provides a decidedly mixed picture on the effectiveness of most SME programs in improving firm-level productivity, expanding markets and creating jobs.

This paper evaluates SME support programs in Chile using a long panel of firm-level data for two groups of firms – a treatment group that participated in SME programs and a control group of firms that did not. In 2004, a random sample of over 600 establishments from six manufacturing sectors provided information about their participation in different support programs and the year of that participation, ranging from the early 1990s up to 2004. The survey elicited information on 7 specific programs providing technical assistance, cluster or network formation, innovation and technology transfer and finance plus 2 open-end “other program” categories. This firm survey was linked to an unbalanced panel of establishments from the annual industrial survey (ENIA) covering the period between 1992 and 2006, providing a wealth of detailed and comparable time series data on establishment characteristics, inputs and outputs, sales, exports, employment, and wages and compensation.

These unique panel data provide an unprecedented opportunity to address several issues that have plagued impact evaluations of SME programs in most countries and in Chile as well. First, the availability of multiple years of information on treated establishments, both before and after program participation, allows us to estimate the impacts of support programs free from selection biases arising from differences between the treatment and control groups in observable attributes and in unobserved heterogeneity. Second, unlike most evaluation studies that track participants for only a year or two after program completion, in our data some treatment firms are observed for as long as 10 years after program participation. Many impacts are only realized over time, which might explain why many impact evaluation studies (including those in Chile) find short-term gains in intermediate outcomes such as training or adoption of new technology and business practices but only mixed evidence of longer-term improvements in sales, productivity or employment growth. Finally, the program participation questions in the firm survey allow us to identify a control group of firms that have never participated in any programs.

2
In many evaluation studies, this is complicated by the presence of a plethora of programs targeting the same universe of firms and lack of centralized administrative records on beneficiaries from all the different programs. One consequence is that some part of the control group may actually have participated in other programs, thus possibly contaminating the counter-factual and biasing impact estimates of the specific SME program under study.

The paper finds a consistent set of results on the impacts of program participation using propensity score matching (PSM) and difference-in-difference (DID) models. First, using PSM models on 2004 cross-section data, we find positive and significant impacts of treatment on intermediate outcomes – increased investments in training, introduction of new products and processes, ISO certification and links with other public and private institutions – as well as final outcomes such as sales, wages and labor productivity, reversing lower average group means of these outcomes in the treatment group. Second, using difference-in-differences (DID) models together with PSM to control for observed and unobserved heterogeneity, we find robust positive and statistically significant program impacts on sales, production, employment, labor productivity and wages. We draw the inference that level estimates of final outcomes may be confounded by unobserved heterogeneity motivating selection into programs of firms with relatively lower productivity levels. Third, we test and find positive outcomes for programs providing subsidized technical assistance, support for cluster formation and technology upgrading but not financing programs. Finally, we find evidence of positive and growing time-effects from program participation, typically between 4-10 years after starting participation for final outcomes such as sales, production and labor productivity but not for employment, wages and exports.

The remainder of this paper is organized as follows. Section II begins with an overview of SME programs in Chile during the mid-1990s and early 2000s and several studies that have evaluated some of these programs. Section III describes the data used in this paper. It provides estimates of program use by manufacturing establishments canvassed in the 2004 Chile Investment Climate Survey (ICS) and compares the characteristics of firms that participated (the “treatment group”) with firms that did not (the “control group”). Section IV describes the estimation approach and reports propensity score matching estimates of the impacts of participation in any SME program on intermediate and final outcomes measured in 2004. Section V focuses on the panel data over the 1994 to 2006 period, using panel regression models to estimate treatment effects on final outcomes measured in levels and in differences. It tests for differences across SME programs, the post-treatment time effects of program participation, and sensitivity analysis bounding the impact estimates for the potential role of support programs in inhibiting exit from the panel by inefficient firms. The final section VI concludes.

3
# **II. Overview of SME Programs in Chile**<sup>**2**</sup>

Chile invests US$400 to US$600 million annually in private sector support programs, ranging from loans and credit guarantees to matching grants for business support services to tax rebates for in-service worker training (World Bank 2004). These different programs cover all sectors of the economy, firm sizes and regions. In late 2001, the loan and credit programs totaled USD 643.5 million, of which the largest share (over 60 percent) was accounted for by a one-time debt restructuring program implemented by CORFO, the economic development agency of Chile, in response to an economic downturn. In that same year, the matching grant and training rebate programs totaled USD 328.7 million. The largest was the training rebate managed by SENCE, the national training authority under the Ministry of Labor, followed by the different business support programs managed by CORFO, the export promotion program of PROCHILE the national export promotion agency, and the innovation fund FONDEF managed by CONICYT, the national science and technology research council.

# **A. Matching Grants and Credit Programs for Industrial Enterprises**

The largest portfolio of matching grants and credit programs covering enterprises in the industrial sector – the focus of this study – is administered by CORFO (Corporación de Fomento de la Producción). Set up in 1939, CORFO’s mission is to advance economic development in Chile by promoting competitiveness and investments, contributing to the generation of jobs for skilled workers, and insuring equal access to services promoting business modernization. The design of CORFO’s programs is guided by considerations of market imperfections and demand. The first principle is that the State should only intervene when there are clear market failures. These include diseconomies of scale, imperfect information about markets and technology, barriers to inter-firm cooperation and limited access to finance, constraints that are especially pertinent to SMEs.<sup>3</sup> In addition, CORFO does not discriminate between economic sectors or geographical regions in the allocation of its resources. All programs should be demand driven, as demonstrated by private sector ownership and co-financing. Projects are funded on the basis of proposals by individual firms or groups of enterprises meeting transparent criteria, typically for two to three years to ensure that support is time-limited. Finally, CORFO outsources the delivery of many programs through public agencies, regional governments, public and private institutes and industry associations, though it directly manages the delivery of some innovation and regional programs through its network of field offices.

Within these guiding principles, CORFO has implemented several major grant and credit programs since the early 1990s to:

> 2 This overview of SME programs in Chile draws heavily upon World Bank (2004), “Chile:  A Strategy to Promote Innovative Small and Medium Enterprises”.

> 3 Chile defines SMEs using an inflation-indexed measure of annual sales or _unidad de fomento (UF)_ . The size cut-off  for “micro” is 2,500 UF (about US$55,000 in March 2004), “small” is 25,000 UF (about US$550,000), and “medium” is 100,000 UF (US$2.2 million).  For comparability with the other country studies, this paper uses an alternative employment-based definition of SMEs (see Section III).

4
- advance technological research and development and technology upgrading

- promote business networking and cooperation especially among SMEs

- facilitate modernization of business practices to increase access to different markets

- support access to finance for new firms, smaller firms and exporting firms, and

- contribute to regional development by stimulating private investment.

**_Fondo Nacional de Desarrollo Tecnológico y Productivo_** (National Productivity and Technological Development Fund). FONTEC operates several financing lines to support development of new products and production processes, overseas missions and consulting for technology acquisition, support for technology transfer centers to adapt and diffuse new technologies, and pre-investment feasibility studies of potentially useful technologies.  In the 10 years since FONTEC was established in 1991, the fund has supported more than 1,700 innovation projects with a value totaling US$ 250 million of which 35 percent was subsidized by matching grants. Over 6,000 firms participated in FONTEC of which 85 percent were SMEs.

**_Proyectos Asociativos de Fomento_** (Group Development Projects). PROFO targets groups of enterprises, and is designed to overcome scale-based barriers such as access to technology, markets and management skills by providing incentives for firms to voluntarily come together in a project to address a common set of production or management problems. This program finances a share of project expenses (on a declining scale) for joint actions, training, market research and product marketing, typically for three or four years. During 2001, CORFO supported 445 projects totaling 16,613 million pesos (approximately US$23 million) of which 36 percent came from PROFO. Since its inception to 2001, over 33,000 enterprises have participated in the program.

**_Programa de Desarrollo de Proveedores_** (Supplier Development Program). PDP seeks to foster vertical linkages between firms, and it offers incentives for larger firms to provide training on quality standards and product design to local small and medium firms so that they can become reliable suppliers. During 2001, there were 82 projects in this program, totaling 2,449 million pesos (about US$3.4 million) of which the government share was just under 60 percent. Of the 3,036 businesses that participated in this program, 94 percent were SMEs.

**_Fondo de Desarrollo e Innovación_** (Development and Innovation Fund).  The mission of the FDI program is to fund innovation and technological change projects in strategically important industries that contribute to both economic and social development. Unlike the FONTEC program that targets enterprises, FDI mainly supports pre-competitive joint technology projects by research centers and enterprises.  In 2002, FDI provided US$10 million in funding to 62 such joint projects.

**_Fondos de Asistencia Técnica_** (Technical Assistance Fund).  FAT is a matching grant program for SMEs that subsidizes the costs of technical assistance to address specific problems including marketing, product design, production processes, information systems and pollution control. Unlike PROFO, FAT is typically used by individual SMEs though CORFO encourages their use by groups of enterprises. While the program started

5
small with just under 350 SMEs in 1994, the use of FAT has grown over time to about 7,000 enterprises annually by the year 2000.

**_Lineas de Financiamientos_** (Lines of Financing).  In addition to its business support programs, CORFO also provides different lines of financing to SMEs through its credit and loan guarantee programs. These include credit lines to SMEs for productive investments - **_Financiamiento de Inversiones de las medianas y pequeñas empresas_** – and SME debt restructuring - **_Reprogramación de deudas PYME_** – a one-time response to a crisis caused by a sudden economic downturn.

**_Programas Territoriales Integrados_** (Integrated Territorial Development Programs). Set up by CORFO in 2000, PTI sees to promote region-based private sector development and productivity growth through the coordinated use of a range of CORFO programs. It combines training activities, innovation, infrastructure, technical assistance, and business and finance networking.  In 2001, the PTI financed nine projects at a total public expense of about US$565,000.

Business support and credit programs are also administered by several other government agencies. The most noteworthy among these are:

**_Servicio Nacional de Capacitación y Empleo_ (** National Training and Employment Service).  SENCE, a government agency under the Ministry of Labor, administers a tax rebate program to upgrade worker skills and thus contribute to employment, improvements in worker and enterprise productivity, and the quality of products and processes. Enterprises using the SENCE incentive are given a tax rebate from the training payroll levy for implementing in-service training programs organized and delivered by a network of registered public and private sector training institutes, universities and centers of technical education. It is estimated that 116,000 enterprises used this training tax incentive in 2002, a dramatic increase from just over 17,000 enterprises in 1988.

**_Programa de Promoción de Exportaciones_** (Export Promotion Program). The National Agency for Export Promotion (PROCHILE), established in 1975, administers the Export Promotion Program to promote Chilean exports and facilitate entry of exporting firms into international markets. In this program, PROCHILE works jointly with export committees comprising four or more enterprises in the financing, design and implementation of international promotion campaigns, market research and feasibility studies, and participation in international fairs.

**_Fondo de Fomento al Desarrollo Científico y Tecnológico_** (Science and Technology Development Fund).  FONDEF, operated by the National Commission on Scientific and Technological Research (CONICYT), funds pre-competitive R&D and technology projects organized jointly by universities, technology institutes and the private sector. From 1991 to 2003, FONDEF invested 97 billion pesos in R&D projects and 4.3 billion pesos in technology transfer projects.

**_Servicio de Cooperación Técnica_** (Technical Cooperation Services). SERCOTEC, founded in 1952, is Chile’s business development agency whose mission is to improve

6
the competitiveness of micro and small businesses. It mobilizes training and technical services to develop managerial skills, promote networking and technology use among SMEs, often in conjunction with regional agencies. SERCOTEC designs and implements its own programs but also acts as an intermediation agent for some of CORFO’s matching-grant programs.

**_Fondo de Garantía para la Pequeña Empresa_** (Guarantee Fund for Small Enterprises). The principal client of FOGAPE is Banco del Estado, a commerically oriented government retail bank which provides loan guarantees for micro and small businesses. The fund of US$50 million guarantees loans for up to ten times the guarantee amount. The average SME loan guaranteed by FOGAPE is 320 UF or about USD 7,500.

# **B. Impact Evaluations of SME Programs in Chile**

Several of Chile’s enterprise support programs – the export promotion progam of PROCHILE, the business networking program PROFO, and the innovation and technology transfer programs of FONTEC – have been rigorously evaluated. The impact evaluation studies used, in common, a non-experimental approach with a treatment group (program participants) and a control group (non-participants). Difference-in-difference (DID) methods were used to address potential biases from time-invariant unobserved firm heterogeneity and propensity score matching to better select treatment and control groups matched on observable firm attributes (these impact evaluation methods are discussed further in Section IV).

Alvarez and Crespi (2000) evaluated the impacts of PROCHILE using a survey of 365 firms drawn randomly from the universe of exporting firms tracked by the Central Bank of Chile – 178 treatment firms that had participated in PROCHILE and 187 control group firms that had not. The survey, covering the period between 1992 and 1996, elicited qualitative information on changes in firm behavior as well as quantitative information on number of exported products, number of destination markets and value of sales. The evaluation results using DID suggested that participation in PROCHILE programs led to qualitative improvements in several dimensions of firm behavior, but mixed results for quantitative outcome indicators. Relative to the control group, PROCHILE participants experienced technological gains (in products, productive processes and organizational forms), more strategic alliances with other companies, improvements in negotiation and access to commercial information, hiring and training of specialized staff, and increased investments in export promotion activities. In quantitative terms, participation in PROCHILE increased by one the number of destination markets of the treatment group relative to the control group. However, there were no significant impacts of participation on the number of exported products or the value of exports (in fact, the control group may have outperformed the participants).

Benavente and Crespi (2003) studied the effects of participation in a PROFO in the early 1990s. They compared a treatment group of 102 SMEs that had completed a three-year cycle of participation in a PROFO by 1995 to a control group of 148 firms of similar size, industry and region drawn randomly from the annual industry survey for the years 1992 7
to 1995. The treatment group was administered a beneficiary survey covering the period between 1992 – the pre-treatment year – and 1995 which marked the end of their participation. The study yielded two principal results. First, participation in PROFOs was associated with improvements in administration, planning and marketing, increased managerial and worker training, and greater access to other public institutions for extension services, consultants, and funds for technology and technical assistance such as FONTEC and FAT; however, only small gains were achieved in introduction of new products or productive processes. Second, participation was associated with gains in total factor productivity (TFP) growth measured by the residual from a Cobb-Douglas production function model. The treatment group experienced higher TFP growth as compared to the control group, ranging between 11 and 14 percent depending upon model specifications with and without propensity score matching.

Benavente, Crespi and Maffioli (2007) investigated the impacts of participation in FONTEC programs on firms’ R&D expenditures, innovation strategies, technological outputs and economic performance. The data comprised 219 firms that had benefited from FONTEC and a control group provided by the Tax Authorities of 220 nonparticipating firms with similar geographical and sector distributions as the treatment group. Both groups were administered a survey questionnaire on experiences with the program and on key qualitative and quantitative outcomes covering the period from 1998 to 2002. Differences between the two groups necessitated a re-matching of the samples using PSM, revealing that younger firms and firms in the more advanced manufacturing sectors were more likely to have participated in the FONTEC program. Using DID methods combined with PSM, they found that participation in FONTEC led to some crowding out of own R&D resources, increased interactions with external sources of knowledge and financing, improved process innovations but not new product development. Although they found positive impacts on employment, sales and exports, there were no significant gains in productivity growth leading the authors to suggest that “R&D activities may take some time to have an impact ... and therefore more time may be needed to obtain conclusive results in terms of productivity.” The principal findings and limitations of previous impact evaluations of SME programs in Chile may be summarized as follows. First, they found evidence that program participation was associated with improvements in intermediate or short-term outcomes but with mixed results on final outcomes which may take time to be realized, suggesting that longer panels are needed to measure program impacts on firm performance. Second, all three studies found increased interactions with, and use of support services from, other public institutions. While this was seen as a positive program outcome, it raises questions about attribution of gains in firm performance between the program being evaluated and other support services used. Finally, the selection of control groups in all three studies may have included some firms that had participated in other programs than the one being evaluated. If so, they potentially bias the counterfactual, and thus the impacts of program participation estimated for the treatment group. Some of these findings and issues are revisited in the following section.

8
# **III. The Chile Data**

The data used in this study come from two sources – the 2004 Chile Investment Climate Survey (ICS) and the _Encuesta Nacional Industrial Annual_ (Annual Industrial Survey) or ENIA, both fielded by the national statistical office INE:

1. The 2004 Chile ICS was commissioned by the World Bank and contains about 1,000 enterprises in five regions and nine sectors, six of which are in the manufacturing sector. In common with other World Bank investment climate surveys, the Chile ICS elicited firms’ perceptions about a wide range of business environment constraints, as well as detailed quantitative information on firm attributes, technology, training, workforce characteristics, wages, and production or sales over the past three years. In addition, it contained a small module of questions on their familiarity with and participation in different government-sponsored SME support programs, and critically the dates of participation in each of the programs used.

2. The ENIA contains information typical of industrial surveys, including firm characteristics (ownership, geographic location, and sector) and quantitative variables such as production inputs and outputs, exports, sub-contracting, fixed assets, employment, wages and other financial data. A panel covering the years 1992-2002 years was provided to the World Bank by INE and was updated to include the years between 2003 and 2006.<sup>4</sup> These panel data allow us to track changes over time in different indicators of firm performance and to estimate the effects of program participation on long-term outcomes such as sales, employment and productivity growth.

Both data sets were linked using a cross-walk variable provided by INE to the World Bank. Responses to the ICS program module were used to identify the treatment and control groups and, from dates of program participation, the pre- and post-program periods in the linked ICS-ENIA panel.

# **A. SME Program Participation**

The Chile ICS contains a sample of 948 enterprises randomly sampled from five regions and nine sectors. Excluding respondents from the information technology (IT) services, biotechnology and aquaculture sectors<sup>5</sup> resulted in a sample size of 603 establishments from six sectors – food and beverages, chemicals, metal products (excluding machinery), machinery and equipment, wood products and paper products.

The 2004 Chile ICS included a SME program module of questions in which firm respondents were asked about seven different CORFO matching grants and credit

> 4 The original 1992-2004 ENIA was updated using a recently available public-use ENIA panel data set covering the period between 1998 and 2006. Establishments in the two panels were matched on the basis of sector, year and production values in the overlapping 1998-2002 period.

> 5 These sectors cannot be linked to the ENIA which only covers the manufacturing sector.

9
programs and two open-ended residual “other program” categories. For each program, respondents provided information on: (i) their familiarity with the program; (ii) whether they were currently participating in the program; (iii) whether they had participated in it in the past and, if so, in what year; (iv) the monetary value of the incentive; and (v) how they rated the importance of the program for their business.<sup>6</sup> The different programs (see Section II for a fuller description of each program) are listed below:

**1.** FAT ( _Fondo de Asistencia Técnica_ ) which provides SMEs with a range of technical assistance services;

2. PROFO ( _Proyecto Asociativo de Fomento_ ) which provides a range of business support services to groups or clusters of enterprises; 3. PDP ( _Programa de Desarrollo de Proveedores_ ) which supports the development and strengthening suppliers for larger enterprises; 4. FONTEC ( _Fondo de Tecnología, Proyectos de Innovación Tecnológica)_ to finance technology development projects jointly with research institutes; 5. FONTEC ( _Fondo de Tecnología, Proyectos de Transferencia Tecnológica)_ to finance technology transfer projects; 6. CORFO ( _Línea de Financiamiento_ ) to provide working capital 7. CORFO ( _Línea de Reprogramación de deuda_ ) to provide capital for debt rescheduling 8. OTHER 1 and OTHER2, open ended “Other programs” not elsewhere listed.

Out of the 603 firms in the ICS sample, 207 reported having participated in one or more programs (henceforth termed the “treatment group”) and 396 stated that they had never participated in any programs (the “control group”). Table 1 reports the distribution of programs participated in by firm respondents and the status of program participation. The table indicates that FAT, PROFO and the technology development line of FONTEC were the 3 programs most commonly used, each accounting for 12-13 percent of the total sample of enterprises. Another 4-5 percent of firms reported using FONTEC’s technology transfer program and CORFO’s credit financing programs, followed by 2 percent each for the PDP supplier development program and CORFO’s debt rescheduling program. The two open-ended program categories accounted for another 4 percent of the total sample but included a wide range of programs each with relatively small sample sizes.<sup>7</sup> Note that the same firms may appear more than once because of multiple program use. While the majority of firms in the treatment group – 63 percent – reported use of just one program, 22 percent used two and 15 percent used three or more programs.

> 6 In this study, we do not use information provided by the treatment group on (iv) the monetary value of the program incentive because of high non-response rates, or (v) the relative rankings of the impacts of program use on the firm.  However, future research might usefully exploit this information to estimate the impacts of differential doses of the treatment, or to test whether firm perceptions of the utility of programs are matched by the realization of outcomes such as those studied in this paper.

> 7 A partial list includes programs from CONICYT’s FONDEF, SERCOTEC, SENCE’s training tax rebate, loan guarantees from FOGAPE, CORFO’s INNOVA and FDI programs, ASEXMA or the Association of Manufacturing Exporters, and SEFOFA or the Society to Promote Manufacturing.  Most programs listed typically only had 1 participant and the largest number had 4 firms. Three firms reported using SENCE’s training incentive  though it is not a program targeting SMEs and, furthermore, is covered elsewhere in the 2004 ICS questionnaire.

10
The rightmost columns of Table 1 show the status of program participation at the time of the 2004 survey. Participation status is characterized as: (1) currently participating in a program, (2) participated in the past and present, and (3) participated in the past. The first category of current participation is relatively small as compared to the other two categories – those that had used programs in the past – a figure which augurs well for getting firm performance data over the post-program period long enough for the potential impacts of program participation to be realized.

A total of 197 out of the 207 establishments in the treatment group provided usable information on the year they started using each program. These date variables were used to define a first-year-of-program-use variable and identify pre-program and post-program years in the ENIA panel. This was straightforward to do for treatment firms that only participated in one program; for firms using multiple programs, this variable was created by comparing the year of participation for all programs reported and selecting the earliest year of participation irrespective of the specific SME program. 10 firms in the treatment group reported first-year dates that preceded 1992, the first year of the ENIA panel, and were dropped since no pre-program information on these establishments are available; two others reported first-year-of-participation dates of 1992. The rest of the treatment group reported dates between 1994 and 2004, 58 between 1992 and 1999, and 127 between 2000 and 2004.<sup>8</sup> For both these latter groups, multiple years of information were available for both the pre-program and post-program periods (in some cases, up to 10 years of post-program information).

Table 2 shows the distribution of the treatment and control group observations in the linked ICS-ENIA panel. Out of 7,292 year-firm observations, 4,772 are for the control group that reported never having participated in any SME programs, and 2,520 are for firms in the treatment group that used one or more programs. The rightmost two columns refer to the treatment group with year-of-first-participation information which was used to define a post-program indicator variable with a value of 0 for all years prior to the first-year-of-participation, and a value of 1 for the first year of participation and all subsequent years. The first column shows the number of year-firm observations in the pre-program participation period ending in 2003, the last pre-participation year for the treatment group surveyed in the 2004 ICS. The second column shows the year-firm observations in the post-program period, with some treatment firms having in excess of 10 years of post-program experience.

# **B. Establishment Characteristics and Outcome Measures**

Together, the linked Chile ICS-ENIA panel data set provides a wealth of information on establishment attributes as well as data on potentially important intermediate (or shortterm) and final (longer-term) outcomes of SME program participation. The data allow us to characterize establishments by their national and foreign capital ownership, geographic location, detailed industry, establishment size, and year in which they started operations.

> 8 This rising trend over time in program use observed in the ICS data is consistent with programmatic data described in Section II of the paper.

11
Table 3 tabulates the distribution of the treatment and control groups across the six sectors and by firm size. We depart from the Chilean classification of SMEs based on annual sales and, instead, define firm size in terms of total employment of permanent and contractual workers – “micro” with 15 or fewer workers, “small” with 16 to 100 workers, “medium” with 101 to 250 workers, and “large” with over 250 workers. The table indicates that the treatment group is well represented in all sectors and firm sizes, making up between one-quarter and one-third of the sample in each sector-size cell.

This observation is pertinent to the impact evaluation studies reviewed earlier for two reasons. It suggests, first, that any firms randomly selected from the underlying population of firms by sector and size to serve as a control group are likely to include a high proportion of previously treated firms. A prior screening of firms for past and current program participation is essential if an appropriate control group of non-program participants is to be selected. Second, as the previous studies themselves note, selecting a control group based on observable attributes such as sector and size is likely to be inadequate; even with similar sector-size distributions, the treatment and control groups can have very different pre-program values of sales, productivity or wages. As such, some of these studies conducted a second round of analysis to select a better matched sample of treatment and control group firms.<sup>9</sup> The linked ICS-ENIA contains rich information on a range of intermediate and final outcomes from participation in SME programs. Viewed through a program logic framework, intermediate outcomes are the short-term changes in firm behavior and practices that the program directly affects through the delivery of technical assistance and business support services and credit; the final outcomes are longer-term improvements in firm performance that firms may realize through program interventions on the intermediate outcomes. The 2004 ICS provides contemporaneous cross-section data on technology inputs and outputs, linkages with other firms and use of quality control practices, worker training and use of the SENCE tax rebate incentive. These intermediate outcomes are some of the same variables that previous evaluation studies have found to be important outcomes of participation in programs such as PROFO, FONTEC and PROCHILE. The ENIA panel contains annual data on measures of firm performance such as sales, gross value of production, employment, total compensation, and income from exports. The long-term outcomes that SME programs seek to influence typically include increased production and sales, entry into export markets, creation of new jobs, higher wages per worker, and improved labor productivity.

Table 4 provides summary statistics on these key outcome measures, comparing the means and differences in means between the treatment and control groups, and t-tests of their statistical significance. Panel A focuses on the intermediate outcome measures elicited in the 2004 ICS:

- (i) _innovation and technology inputs_ – have a foreign technology license, acquired new technology over the last 2 years, does own research and development (R&D) or via third parties, and installed new electric and numerical control (NC) equipment and machinery;

> 9 See Benavente and Crespi (2003) and Benavente, Crespi and Maffioli (2007).

12
- (ii) _technology outputs_ – introduced new product lines and introduced new production processes, both in the last two years;

- (iii) _Inter-firm linkages and quality control_ – membership in an industry association, and have or getting ISO and other internationally-recognized quality certification;

- (iv) _worker training_ – firm provided employees with in-house training, with training through an external provider, used SENCE tax incentive to finance training, and the percentages of skilled and unskilled workers that received training last year.

Panel B reports the annual mean values of several final outcome measures, for the 2004 cross-section and over the 1992 to 2006 period. They include (in logarithms) income from sales, gross value of production, total employment, average total compensation per worker, and production per worker or labor productivity, all expressed in real 1996 pesos.<sup>10</sup> In addition, Panel B includes an indicator for exports as well as a continuous measure of exporting income as a percentage of sales.

Several points emerge from Table 4 when the mean values of intermediate outcomes are compared for the treatment and control groups. First, with one exception (having foreign technology licenses), firms in the treatment group as a whole have higher mean values of each of the intermediate outcomes relative to the control group. T-tests indicate that some of these group differences are statistically significant at the 1 percent level – doing R&D, introduction of new product lines, having ISO or other internationally recognized quality certification, and all training measures. It appears that program interventions are having the desired impacts on intermediate outcomes of firms in the treatment group.

Second, and in contrast to the previous results, the final outcomes in 2004 for the treatment group are decidedly mixed, with lower mean values for sales, wages and labor productivity but higher means for employment and export sales in the treatment group as compared to the control group. However, only the differences in labor productivity are significant at the 5 percent level. This pattern is repeated for a broader set of final outcome measures averaged over the 1992 to 2006 period, but this time all differences in group means are significant at the 1 or 5 percent levels. This pattern appears to be counter-intuitive, suggesting that program participation was associated with lower levels of most measures of firm performance. An alternative explanation is that SME support programs tend to attract poorly performing firms with lower than average unobserved productivity attributes, but that program participation improves their performance relative to what might have prevailed had they not.<sup>11</sup> This hypothesis is discussed further in the following section.

> 10 Nominal values of sales, production, value added and wages reported in current pesos were deflated to constant 1996 pesos using producer price indices and the consumer price index, respectively. 11 Benavente, Crespi and Maffioli (2007) made the same point in re-selecting their treatment and control groups matched on the basis of sector, size and region because of wide differences in the pre-treatment performance of the original treatment and control groups.  Tan and Lopez-Acevedo (2007) also found similar counter-intuitive results on performance of treatment and control groups in their evaluation of SME programs in Mexico.

13
# **IV. Empirical Approach and Initial Findings**

Consider a general model for firm _i_ in time _t_ which relates outcomes Y (such as sales or employment) to observable firm attributes X (such as firm size and sector) and an indicator variable for participation in a program D: where  is made up of a time-invariant firm-specific component _v_ and a randomly distributed error term _u_ . The impact evaluation challenge is to estimate the net impacts of program participation  free of bias from self-selection of firms into programs based on their observable and unobservable productivity attributes. The evaluation studies reviewed earlier sought to address these potential selection biases through the combined use of propensity score matching and difference in difference methods. We adapt these same approaches to accommodate the specific nature of our panel data.

Figure 1 illustrates the evaluation challenge by graphing the time-path of outcome Y for a firm – treated firm 1 – that participates in a program in time 0. The impact of program participation is the improvement in Y subsequent to time 0 (the dashed line or Y<sup>1</sup> ) relative to what Y would have been in the absence of the program. Because this counterfactual is not observed, the analyst must rely on the time-path of Y<sup>0</sup> (the solid line) of a control group of similar firms; in many evaluation studies, this control group is selected from the universe of firms to match the treatment group in its distribution of attributes by sector, firm size and geographic location.

# **A. PSM and DID Methods**

Recent studies have matched the treatment and control groups on the basis of a propensity score. Rosenbaum and Rubin (1983) define the propensity score as the conditional probability of receiving a treatment: where, as before, D={0,1} is an indicator variable for program participation, and X is a multi-dimensional vector of pre-treatment attributes of firms. They show that if exposure to treatment is random within cells defined by X, it is also random within cells defined by the values of p(X). This allows us to write  , the average treatment effect on the treated (ATT), as: that is, the expected difference in outcomes (Y<sup>1</sup> – Y<sup>0</sup> ) between the treatment and control 14
groups matched by their propensity scores. The propensity score p(X) can be estimated from a logit or probit model of program participation regressed on a vector of preparticipation attributes of the two groups.

Propensity score matching (PSM) may not be enough if self-selection into programs is also based on productivity attributes not observable to the analyst. Figure 1 shows the case of a second treated firm with similar observable pre-participation attributes as the control group but with pre-participation values of Y lower than that of the control group by an amount _v_ , assumed to be time-invariant. While the program improves Y<sup>1</sup> (the bold dashed line) relative to pre-participation levels, post-treatment Y<sup>1</sup> is lower than Y<sup>0</sup> of the control group even though the gap between them (Y<sup>1</sup> -Y<sup>0</sup> ) diminishes over time. The presence of unobserved attributes _v_ can thus bias estimates of  , even yielding negative program impacts as this example illustrates.

The confounding effects of _v_ on  can be addressed through difference in difference (DID) methods. Let T= 0 and T=1 represent the pre- and post-participation periods. First differencing equation (1) for the treatment group and the control group eliminates the time invariant _v_ term: We extend these analytic approaches to accommodate the specific panel structure of our panel data. Many program impact evaluations such as the studies reviewed earlier track a single cohort of treated firms and their control group from pre-participation to program completion, typically over a 3-5 year period. This simplifies the impact evaluation considerably: estimate a logit model of program participation on pre-participation attributes to calculate the propensity score, and then use PSM to compare mean postprogram outcomes of the two groups measured either in levels or in first differences (between pre- and post-program outcomes). In contrast, our panel data track successive cohorts of treated firms (and their control group) entering SME programs between 1992 and 2004. Time since program use also varies in our data, whether post-program outcomes are compared in 2004 or at any point in time over the 1994 to 2006 period.

We use a Cox proportional hazard model to estimate the propensity score of the 15
likelihood of program participation for the sample of treatment and control groups followed over the 1994 to 2004 period.<sup>12</sup> The Cox proportional hazard model relates the likelihood of entry into a program, conditional upon survival (non-entry) up to that point in time, to a baseline hazard function and a set of independent covariates. The underlying hazard function _h_ ( _t_ ,..) may be written as follows: where _Z_ is a vector of _m_ covariates, and _h_ 0 (<sup>_t_</sup> ) is the baseline hazard when the values of all the covariates are set to 0. This model can be made linear by dividing both sides of equation (6) by _h_ 0 (<sup>_t_</sup> ) and taking natural logarithms: This leaves an equation (7) that is readily estimable, and from which the predicted value of  . _Z_ can be calculated.  . _Z_ is the relative hazard of program entry for firms with attributes _Z_ , and we use it as the propensity score for defining the region of common support for matching successive cohorts of treated firms and their control group.

The multitude of treatment cohorts also poses challenges for traditional PSM methods of estimating impacts. This will become evident when we use cross sectional data from the 2004 ICS to investigate the impact of program participation on levels of intermediate and final outcomes measured at one point in time.<sup>13</sup> In 2004, measured outcomes from the different treatment cohorts reflect wide variations in time since program participation, some as long as 10 years ago, as well as the cumulative effects of macroeconomic shocks occurring over this period. PSM methods cannot readily accommodate variations in time following program participation by the different treatment cohorts, or separate treatment effects from broader time-related effects affecting both groups. This is compounded if treatment effects are only realized over time. We flag this caveat but defer the issue to analyses using the panel data.

# **B.  Conditional Likelihood of Program Participation**

We begin by estimating a Cox proportional hazards model to calculate the relative hazard rate of program participation of firms in our panel based on their pre-participation attributes. The sample included treated and control group firms in the 1992 to 2004 panel, and firms that entered the panel sometime over this period. Entry into a program

> 12 An alternative approach is to estimate separate logit models of program participation for different crosssections (or year intervals) to derive propensity scores for each treatment cohort (or groups of cohorts). This did not prove feasible because of small sample sizes which led to very imprecise estimates of the logit model.  The Cox proportional hazards model was preferred not only because of sample size considerations but also for its unified treatment of the underlying process of selection into programs over time.

> 13 Here, the intermediate outcomes are measured in levels, and not first differences, since they were elicited for just one point in time by the 2004 Investment Climate Survey (ICS).  The subsequent analysis of final outcomes compares the level and DID estimates of treatment effects using the linked ICS-ENIA panel data.

16
(the failure event) is captured by a post-program variable, defined earlier, with a value of 0 for all years prior to the first-year-of-participation, and a value of 1 for the first year of participation and all subsequent years.

We restricted the treatment group to firms with first-year program participation dates starting in 1994 or later, ensuring that the treated firms have at least two years of preprogram information. This exclusion was motivated by the potentially important role that unobserved time-varying productivity factors, such as anticipated sales growth or export growth opportunities, might play in shaping firm decisions to participate in SME programs. To reflect this possibility, we construct two variables – the 1 year lag of the logarithm of sales, and the change in the logarithm of sales between time t-1 and t-2. Other things equal, we hypothesize that the likelihood of program participation is higher among firms with low values of lagged sales (indicating relatively poor performance) but with good prospects for future growth.

In addition to these two variables, we included several other potential covariates of program participation, all lagged one year. First, we control for the effects of total employment on participation using indicator variables for small, medium and large firms. Some, but not all programs, target SMEs and we would generally expect participation to rise with firm size relative to micro firms except in the largest size category. Second, we capture the effects of years in operation through indicator variables for establishment in the 1980s and 1990s, measured relative to the oldest firms established in the 1970s or earlier. Other things equal, we would expect a greater likelihood of program participation among younger, less experienced firms. Third, we include an indicator variable for foreign capital ownership, expecting a lower likelihood of participation because of the possibility of getting assistance from foreign partners. Finally, we include indicator variables for 5 industrial sectors (food and beverages as the omitted group), and location in the national capital region of Santiago (region 13). We are agnostic about how they influence program participation.

The results of estimating the Cox proportional hazards model are reported in Table 5. In the Cox model, estimated hazard ratios greater than 1 are associated with an increase in the likelihood of program participation, while hazard ratios less than 1 are associated with a decreased likelihood. Many covariates have the expected effects on the likelihood of participation but statistical significance is only attained for several independent variables. First, larger firms are more likely to participate than micro firms. Second, the differences in likelihood across sectors are small, the exception being the wood products sector which is less likely to participate. Third, while older firms and firms with foreign capital ownership are less likely to participate, these effects are not statistically significant. Fourth, firms located outside the capital region are significantly more likely to participate, reflecting either a greater demand for business support and credit services in remote areas or more active program outreach to outlying regions by program administrators. Finally, consistent with our priors, firms with lower lagged sales but good growth prospects are more likely to participate in programs, though only lagged sales are statistically significant.

17
The Cox results are used to predict the relative hazard rates for the treatment and control groups. As the propensity score for each firm, we use the mean of their hazard rates for all years in which they are available.<sup>14</sup> The hazard rate averaged 0.124 for the treatment group and 0.099 for the control group, consistent with the treatment group as a whole having a higher relative probability of program participation. Figure 2 graphs the distributions of these relative hazard rates for the treatment and control groups, and the region of common support.<sup>15</sup> The graph shows the treatment group to have greater density in the upper tail of the distribution of propensity scores than the control group. Nonetheless, within the region of common support which is quite wide, every firm in each group has a positive probability of participating in SME programs though some may have higher probabilities than others.

# **C. Estimating Program Impacts in 2004**

With propensity scores in hand, we turn to an investigation of the impacts of program participation on levels of intermediate and final outcomes in 2004. The definitions and summary statistics of these outcome measures were reported previously in Table 4, separately for the treatment and control groups. That table highlighted the fact that the treatment group generally had higher mean values of intermediate outcomes as compared to the control group, but generally lower levels of most final outcomes measures.

We estimate the average treatment effect of program participation using the nearest neighbor matching estimator. The nearest neighbor estimator essentially compares the treated firm to an untreated firm from the control group with the most similar propensity score, and is an estimator that is commonly used when sample sizes are small.<sup>16</sup> The analysis is restricted to the treatment and control group firms in the region of common support. Table 6 reports the means of each outcome for the matched treatment and control groups, the differences in means, and t-tests of the statistical significance of these differences. Panel A focuses on the intermediate outcomes elicited by the 2004 ICS, Panel B on selected final outcomes computed from the 2004 ENIA.

Panel A of Table 6 reveals that program participation is associated with higher means values of the intermediate outcomes, though these differences in means are statistically significant only for some outcomes. Note that we cannot infer a causal relationship from program participation to outcomes since no pre-participation measures are available. Nonetheless, the patterns observed here are consistent with those found in other impact evaluations of Chilean programs that did have pre- and post-participation data on these variables. Compared to their matched control group, treated firms are significantly more likely to be involved in R&D, to have introduced new product lines and production

> 14 For the treatment group, the means are computed for all years up until the year of program participation, after which relative hazards rates are not defined because the failure event has occurred.

> 15 The region defined by the maximum and minimum values of the propensity scores of the other group. Since the minimum and maximum values for the treatment group was 0.0488 and 0.2932, and 0.0296 and 0.2740 for the control group, the region of common support lies between 0.0488 and 0.274. Some firms in each group fall outside the region of common support – 3 treatment firms with high hazard rates, and 16 control group firms with low hazard rates.

> 16 Other PSM estimators include caliper and kernel matching, but these were not explored in the paper.

18
processes, have or are currently in the process of getting ISO and other internationally recognized quality certification, and providing its workers with training. The training results are especially significant, with treated firms providing a higher proportion of its skilled and unskilled workers with training, both in-service and externally, and financing training using the SENCE tax incentive.

Panel B also indicates that program participation is associated positively with higher mean values of several final outcomes. Relative to the matched control group, treated firms have significantly higher levels of sales, share of exports in sales, employment, and compensation per worker; however, no significant differences were found for labor productivity. These results are especially striking when compared to the unmatched group means of the treatment and control groups reported in Table 4. There, the treated firms had higher means of employment and exports, but lower mean values of sales, wages and labor productivity.

How do we interpret these sign reversals? They suggest that the confounding effects of unobserved heterogeneity on treatment effects <u>estimated in levels</u> can be overcome in part through improved matching of treated and control firms based on their propensity scores. Recall the results in Table 5 from estimating a Cox proportional hazards model to calculate propensity scores for the treatment and control groups. To the extent that unobserved firm heterogeneity is correlated with these observed pre-treatment attributes, matching based on the propensity score can control for some part of this unobserved productivity differences. It is also worth noting that the propensity score index gives weight to pre-treatment sales and sales growth, in effect mimicking the difference-indifference (DID) comparisons of pre- and post-treatment outcomes discussed next.

19
# **V. Estimating Program Impacts Using the ICS-ENIA Panel**

We now turn our attention from the cross-section to the linked ICS-ENIA panel to more fully exploit the availability of annual data on a range of final outcomes. Our objective here is to estimate the longer-term impacts of program participation controlling for the effects of observed and unobservable productivity attributes, and to test for differences in the treatment effects of four broad types of support programs – business development services (BDS), cluster programs, technology development, and credit programs. Also of interest is investigating the time effects of program participation, and how quickly or slowly program impacts are realized over time. Finally, we are interested in testing the sensitivity of our program impact estimates to the possibility that program participation inhibits firm exit from our panel data.

As noted in the previous section, traditional PSM methods are not well set up to accommodate the specific structure of our data. In this section, we adopt a more flexible regression approach that allows us to estimate treatment effects taking into account differing entry points into programs, use of multiple types of programs, widely varying time since program participation, and year specific shocks. We rely on fixed-effects models to eliminate the effects of unobserved firm heterogeneity as a source of bias in estimates of program impacts. In the spirit of the PSM approach, we continue to focus on the matched sample of treatment and control group firms in the region of common support identified by their propensity scores.

Consider an expanded equation (1) in levels: which includes the program indicator _D_ , an interaction term between _D_ and a variable _YRS_ measuring years-since-first-participated in the program, and the time-invariant error term<sup>_v_</sup> _i_<sup>. Estimating equation (8) in levels is likely to lead to biased estimates of</sup> because of the omitted variable<sup>_v_</sup> _i_<sup>, with the direction of bias being determined by the</sup> correlation between<sup>_v_</sup> _i_<sup>and Di. When the correlation is negative, as in the case illustrated</sup> in Figure 1 where less productive firms are attracted to government support programs, estimates of  are likely to be biased downwards. The fixed effects estimator addresses this possibility by taking deviations from variable means so that equation (8) can be rewritten as: where firm variable means are denoted by a single subscript _i_ . Like first differencing, the fixed effects transformation eliminates the potentially confounding effects of<sup>_v_</sup> _i_<sup>.</sup> This framework allows us to address several issues. First, are estimates of program impacts potentially biased by unobserved firm heterogeneity? We compare treatment effects estimated from a levels model (equation 8) and a fixed-effects model (equation 9) 20
to test for potential biases in estimates of treatment effects from unobserved firm heterogeneity. Second, are program impacts larger in some programs than in others? In place of _D_ , an indicator for participation in any program, we include indicator variables for participation in different types of SME programs D1i, D2i ... Dni,, and test for differences in their separate impacts on outcomes. We note that this specification allows for (but does not explicitly model) multiple program use since each program used by firm _i_ have their own program start dates. Finally, how long does it take for program impacts to be realized? We test for time-effects of program impacts from  2, the estimated coefficient on the _YRS_ interaction with _D_ . An alternative, which we use, is to specify _YRS_ as a set of discrete time intervals to allow for non-linear time-effects of program participation.

We use a parsimonious model specification designed to facilitate comparison across different regressions. Eight final outcome measures are selected for study – sales, value added, production, total employment, wages per worker, labor productivity, income from subcontracting and exports. These outcome measures are related to program indicator variables which take on a value of 0 for all years preceding the first-year of participation (pre-program period), and 1 for all years that follow including the first-year (postprogram period). In addition to the program variable(s), our explanatory variables include indicator variables for location in the Santiago capital region, firm size (small, medium and large relative to the omitted micro firm), and year dummy variables for 1995 through 2006 to control for the effects of year-specific stochastic shocks.

# **A. Program Impacts in Levels and Differences**

Table 7 reports the estimated program impacts on eight outcome variables for two specifications of the program participation variable – one for participation in any programs, a second for participation in each of the four different program types. Before turning to the treatment effects, we note (but do not report in the table) the results for the other control variables, namely that most outcomes are higher for firms located in the capital region and for larger firm sizes, and rise over time.<sup>17</sup> Panel A shows the treatment effects on outcomes measured in levels. What emerges is that the average treatment effects estimated for many outcomes measured in levels are negative, and often statistically significant. This counter-intuitive result of negative impacts persists when treatment effects are estimated by type of program. The one exception – the technology development programs of FONTEC – showed positive and statistically significant treatment effects on value added and wages, but the results appear to be implausibly high.

Panel B reports very different treatment effects estimated using the fixed-effects model. First, the average treatment effects of participation in any program changes sign, and are now positive and statistically significant at the 1 or 5 percent level for all outcomes except employment. The average treatment effect on sales and output are broadly similar at 9 percent, the effects on wages and labor productivity are 8 and 7 percent, respectively, and just over 2 percent for the share of exports in sales.<sup>18</sup> Treated firms on average tended

> 17 The full regression results are available upon request from the author.

> 18 We note that the export share regression is more appropriately estimated as a tobit model.  However, the

21
to have relatively higher pre-treatment employment levels as compared to the control group, and keeping employment levels constant may have permitted gains elsewhere in wages and labor productivity.

In Panel B, differences across programs in treatment effects emerged from the analysis by program types. Participation in technical assistance programs (FAT) is associated with the largest and most consistent gains in most outcomes, ranging between 18 and 20 percent for sales and output, 16 percent in labor productivity and 8 percent in wages. Participation in cluster programs (PROFO and PDP) produced gains of 7 to 8 percent in sales, output and wages, while technology development programs yielded gains of about 5 percent in wages and exports as a share of sales. However, use of credit and loan programs was not associated with any improvements in outcomes; if anything, these treated firms had marginally lower sales growth than the control group. It appears that improved access to finance by itself is insufficient to spur firms to make the necessary organizational and technological changes to improve future performance.

From these results, we conclude that unobserved firm heterogeneity motivating the selfselection of less productive firms into treatment can bias downwards estimates of program impacts. We observe this in the case of Chile, and speculate that similar behavioral processes are in operation in other countries and may be responsible in part for the mixed findings of program impact evaluations.<sup>19</sup>

# **B. Time-paths of Treatment Effects**

Thus far, we have estimated the average treatment effect of program participation without consideration for whether these effects vary over time. While useful, these estimates of average treatment effects leave unanswered the question of whether impacts are realized immediately or only slowly over time. In the program logic model, program interventions impact final outcomes only indirectly; they target a range of business development and credit services to address and resolve identified enterprise shortcomings. The resulting impacts on intermediate outcomes – such as new management, marketing and quality systems, worker training, technology development, and greater linkages with other enterprises – in turn give rise (at least in theory) to improvements in final outcomes such as sales, job creation, wages and labor productivity, and exports. How time dependent these impacts are makes a difference for how long the time horizon of impact evaluations should be to reasonably expect measurable impacts on final outcomes.

We test for time-effects of participation in any program by including interactions terms between the program participation measure and time since entering a program, as in equation (9). Rather than forcing a functional form on these time effects (for example, with a quadratic specification of time and time squared), we define a set of indicator variables for different intervals – 1, 2, 3, 4-5, 6-7, 8-10 and over 11 years – following the

> key results are essentially unchanged when a random effects tobit model was estimated.

> 19 Tan and Lopez-Acevedo (2007) came to a similar conclusion in their impact evaluation study of SME programs in Mexico, where they also found sign reversals of treatment effects estimated in levels and in difference-in-differences (DID).

22
date of entering the program. This allows the effects of the interaction terms between the program indicator and time since participation to vary non-linearly with time in (some SME programs can last 2-3 years) and after the program.

The resulting estimates can be interpreted as the time effects of treatment if several assumptions hold. First, these effects are estimated holding constant all other factors that are time varying, including inflation and macroeconomic shocks. The model accounts for these factors by deflating all outcome measures into constant 1996 prices and including year dummy variables to capture year-specific stochastic shocks. A second assumption is that self-selection into treatment is not dependent upon time. The presence of cohort effects in treatment – firms that choose to participate early are different from those that join in later years – can introduce bias into these estimates. For example, if early cohorts were more entrepreneurial (or “go-getters”) than those that came later in the program, then this could bias the results towards finding treatment effects increasing with time since identification of interaction effects comes from comparing at a given point in time firms that had the treatment for different numbers of years.

We checked for, and found no, systematic time-varying differences between treatment cohorts. Two dimensions of treatment cohorts were considered – pre-treatment sales growth experiences and rankings of the importance of program(s) to the establishment.<sup>20</sup> The anticipated sales growth variable, first used to estimate the Cox proportional hazards model, could vary systematically by cohort and lead to bias in the estimated time effects if this ex-ante measure was realized in future growth in sales and other outcomes. Similarly, bias could arise if the value of the treatment varies systematically by cohort, as judged by ex-post rankings of the importance of programs to treated firms. Table 8 reports means of both measures by year of program entry. Sales growth by cohort varies for the most part between 5 and 7 percent, without any systematic trend over time.<sup>21</sup> Similarly, the ranking by cohorts shows no time trend, and its variation around 2.6 suggests that most cohorts judge program impacts to be important to very important.

Table 9 reports estimates of the time-path of program impacts on different final outcomes. As before, we estimated the regression models in levels and in differences, but only report the DID results since no significant time-effects were found for the levels regressions. Consistent with our priors, the results demonstrate that the treatment effects on final outcomes can take a long time to be realized. Compared to the results in Table 7, the coefficient on the participation variable is still positive, but now lower in magnitude and not statistically significant except for the wage outcome. None of the indicator variables for time since participation are statistically significant before 4 years. Beginning with 4-5 year after program entry, the estimated coefficients become positive and increase in value and statistical significance. Using the example of sales, the

> 20 The sales growth variable is the change in log(sales) between time t-2 and t-1, where t is the year of program entry, and it is calculated directly from the ENIA panel. The qualitative rankings of programs on a scale of 1 (minor) to 4 (crucially important) are taken from the 2004 ICS. Since treated firms can (and do) participate in multiple programs, we calculate the weighted mean ranking by year of treatment, using the frequency or each rank cited for a given treatment year as weights..

> 21 This was confirmed by a simple regression of the change in log(sales) on a time trend, with the time coefficient being indistinguishable from 0.

23
treatment effect is 10 percent at 4-5 years, rising to 15 percent at 6-7 years and to 30 percent from 8 years on since program entry. At 6-7 years, the impacts on wages and labor productivity are 10 and 17 percent, growing to 15 and 28 percent respectively by the 11<sup>th</sup> year. While there are no discernable time effects on employment, both existing workers and their employers benefit from these support programs through improved wages and gains in labor productivity exceeding wage costs.

These results can be used to compare the post-program time paths of final outcomes for both groups of firms. Assuming that the treated firms enter SME programs in 1994, we can compute the values of each outcome for each year between 1994 and 2006 using the estimated coefficients of the constant term, program participation variable, indicator variables for time since program entry, and year dummies, interpolating where necessary using the mid-points of each time interval. For comparison, we calculate the corresponding time paths for the control group excluding all the coefficients associated with program participation.

Figure 3 graphs the time paths of output, employment, wages and labor productivity predicted for the treatment and control groups. The treatment effects at any point in time are represented by the gap between the time profiles of the two groups. Take the case of the value of output. Between 1994 and 1997, the time profiles of this variable for the treatment and control groups are barely distinguishable from each other. Subsequently, the gap between them – the treatment effect – widens up to 2004 after which both profiles turn downwards. In the case of employment, the time profiles of labor for both groups decline slightly between 1994 and 2000; after 2001, employment in the treatment group actually rises above that of the control group (which remains roughly constant) yielding positive treatment effects. The treatment effects on labor productivity resemble those of the output variable, not surprisingly since labor productivity is defined as output per worker, and treated firms saw gains in output without increases in employment up until 2001. The treatment effects on wages are initially small but become larger at about 6-7 years after program entry, that is, from 2000 on.

# **C. Bounding Estimates of Program Impacts**

As a final exercise, we investigate the sensitivity of our estimates of treatment effects to a peculiar feature of our data, namely that only ENIA firms that responded to (survived until) the 2004 ICS were included in our panel data. As such, there is firm entry as new firms join the treatment and control groups sometime over this 1992 to 2004 period, but no firm exit from the panel. One implication is that the firms in our panel may not be representative of the firms in the manufacturing sector, being a selected group that had survived (in some cases) up to 14 years in the panel.

How serious is the potential bias from not accounting for firm exit? Using time series ENIA data from the 1980s and 1990s, Alvarez and Vergara (2007) estimated 5-year firm survival rates of about 69 percent, ranging from 67 percent for small firms to 82 percent for large firms. This implies an average annual exit rate of between 7 and 8 percent for the manufacturing sector as a whole. However, the relevant figure here is the differential 24
exit rates for the treatment group as compared to the control group. Fortunately, we have estimates of the relevant exit rates since our panel data track both groups of firms beyond 2004 to 2006, and we observe exit over these two years. The two-year survival rates of 14 percent and 13 percent for the control and treatment groups, respectively, correspond to annual exit rates of 7.5 and 6.5 percent which are broadly similar to the Alvarez and Vergara estimates. Thus, program participation appears to inhibit exit of treated firms by about 1 percent per annum relative to the control group. Since the mean time since program entry is 6 years in our panel data, one possible strategy for addressing this potential bias is to estimate program impacts after trimming the bottom 5 percent of the treatment group in each outcome, assuming that the least well-performing treated firms would have exited in the absence of the program. As a robustness check, we also estimate treatment effects when we trim the bottom 10 percent of the treatment group.

We implemented this sensitivity analysis by sorting and dropping the bottom 5 percent (or 10 percent) of the treatment group’s distribution of each outcome variable. The results of estimating the treatment effects with trimming are reported in Table 10, panels A and B for the 5 and 10 percent exclusions, respectively. Several points emerge from comparing the treatment effects of any program participation estimated with trimming and the original estimates reported in Table 7, panel B. First, the program impacts on sales and output are broadly similar – averaging between 8 and 10 percent as compared to the original estimates of 9 percent. Second, the treatment effects for employment are now positive and marginally significant whereas the original estimates showed no statistically significant impact. Third, wage effects are essentially the same at 7 percent as compared to 8 percent estimated previously. Finally, the biggest change is seen in the effects on labor productivity, which at 8 percent are now just half as large as the 16 percent originally estimated.

Table 10 also reports the treatment effects estimated with trimming by type of program. The broad patterns observed in Table 7 of differential impacts on outcomes by program type are repeated here when outcomes are trimmed. The principal difference is that trimming the bottom 5 percent appears to reduce the treatment effects of BDS programs (FAT) and increase the estimated treatment effects for the cluster programs (PROFO and PDP); trimming the bottom 10 percent further magnifies these differential impacts on the two types of programs. The essential point to take away from this sensitivity analysis is that the direction and magnitudes of treatment effects are broadly robust to controls for potential biases from firm exit.

25
# **VI. Summary and Concluding Remarks**

In this paper, we have evaluated the impacts of SME support programs in Chile using a non-experimental approach, with a treatment group that participated in SME programs and a control group that did not. The unique firm-level panel data used provided an unprecedented opportunity to address several issues that have plagued impact evaluations of SME programs – selectivity bias from observed and unobserved firm heterogeneity, identification of an appropriate control group, and inability to track the treatment group over a long enough period of time for firm performance outcomes to be realized.

Using propensity score matching combined with difference-in-differences models, the paper found evidence that program participation is causally related to improvements in a range of intermediate outcomes (training, adoption of new technology and organizational practices), as well as positive gains in sales, labor productivity, wages and to a lesser extent, employment. Positive treatment effects were also found by type of program, with participation in FAT and PROFO, and to a lesser extent, FONTEC having the most consistent positive impacts on several final outcome measures. No significant treatment effects were found for use of credit and loans programs alone, suggesting that access to finance alone is unlikely to spur firms to make needed organizational and technological changes to improve performance.

These findings provide insights into the mixed results of many previous impact evaluations of SME programs, in Chile as well as in other countries. First, they suggest that negative or insignificant estimates of treatment effects may often be the result of inadequate controls for the self-selection of firms into SME programs. The combination of approaches used in this paper – propensity score matching on the basis of observed pre-treatment attributes, and difference-in-difference methods to account for unobserved heterogeneity – can be quite effective is addressing this issue. Second, the absence of treatment effects may also be the result of a long gestation period for improvements in firm performance to be realized. Our finding of important time effects suggests that a longer time horizon – beyond the one or two years after treatment that is common in many evaluation studies – should be considered in program impact evaluation studies.

The research also suggests areas where the monitoring and evaluation of SME programs could be improved in Chile. There is currently no centralized repository of beneficiary data from all the different programs in Chile despite the government’s commitment to rigorously evaluating program impacts. Absent such a data base, policymakers have little information on specific and/or multiple programs used by different firms; program evaluators are also constrained in identifying an appropriate control group of firms that have never participated in any of these programs. Unless screened out, the inadvertent inclusion in the control group of firms that have participated in other programs could confound estimates of the program impacts of interest. It might also be useful to conduct periodic joint assessments of programs to compare their relative effectiveness and to identify as candidates for closure poorly performing programs or programs that have outlived their usefulness.

26