metadata: authors: 0: author: Cravo,Tulio A.
1: author: Piza,Caio
count: World
docna: 0: docna: The impact of business support services for
            small and medium enterprises on firm performance in low- and
            middle-income countries : a meta-analysis
docty: Policy Research Working Paper
subtopic: Taxation & Subsidies,Labor Markets
teratopic: Finance and Financial Sector Development,Macroeconomics and Economic Growth,Social Protections and Labor,Private Sector Development
abstracts: cdata!: Interventions designed to support small
            and medium enterprises are popular among policy makers,
            given the role small and medium enterprises play in job
            creation around the world. Business support interventions in
            low- and middle-income countries are often based on the
            assumption that market failures and institutional
            constraints impede the growth of small and medium
            enterprises. Significant resources from governments and
            international organizations are directed to small and medium
            enterprises to maximize their socioeconomic impact.
            Business-support interventions for small and medium
            enterprises in low- and middle-income countries most often
            relate to formalization and business environments, exports,
            value chains and clusters, training and technical
            assistance, and access to credit and innovation. Very little
            is known about the impact of such interventions despite the
            abundance of resources directed to small and medium
            enterprise business-support services. This paper
            systematically reviews and summarizes 40 rigorous
            evaluations of small and medium enterprise support services
            in low- and middle-income countries, and presents evidence
            to help inform policy debates. The study found indicative
            evidence that overall business-support interventions help
            improve firm performance and create jobs. However, little is
            still known about which interventions work best for small
            and medium enterprises and why. More rigorous impact
            evaluations are needed to fill the large knowledge gap in
            the field.
repnme: repnme: The impact of business support services for
            small and medium enterprises on firm performance in low- and
            middle-income countries : a meta-analysis
display_title: The impact of business support
            services for small and medium enterprises on firm
            performance in low- and middle-income countries : a meta-analysis
listing_relative_url: /research/2016/05/26360541/impact-business-support-services-small-medium-enterprises-firm-performance-low--middle-income-countries-meta-analysis
topicv3: Job Creation and Job Opportunities
            m1326483 1170,Jobs m1503931 2670,Labor Productivity m1504060 2756,Jobs Strategies m1504080 2769,Tax Simplification m1504614 2527
publishtoextweb_dt: 2016-05-09T00:00:00Z
historic_topic: Finance and Financial Sector Development,Macroeconomics and Economic Growth,Social Protections and Labor,Private Sector Development
no_of_pages: 57
author_keys: 0: author_key: 1239892
model_extractions: WPS7664 Policy Research Working Paper 7664 The Impact of Business Support Services for Small and Medium Enterprises on Firm Performance in Lowand Middle-Income Countries A Meta-Analysis _Tulio A. Cravo Caio Piza_ Development Research Group Impact Evaluation Team May 2016
Policy Research Working Paper 7664

# **Abstract**

Interventions designed to support small and medium enterprises are popular among policy makers, given the role small and medium enterprises play in job creation around the world. Business support interventions in low- and middle-income countries are often based on the assumption that market failures and institutional constraints impede the growth of small and medium enterprises. Significant resources from governments and international organizations are directed to small and medium enterprises to maximize their socioeconomic impact. Business-support interventions for small and medium enterprises in low- and middle-income countries most often relate to formalization and business environments, exports, value chains and clusters, training and technical assistance, and access to credit and innovation. Very little is known about the impact of such interventions despite the abundance of resources directed to small and medium enterprise business-support services. This paper systematically reviews and summarizes 40 rigorous evaluations of small and medium enterprise support services in low- and middleincome countries, and presents evidence to help inform policy debates. The study found indicative evidence that overall business-support interventions help improve firm performance and create jobs. However, little is still known about which interventions work best for small and medium enterprises and why. More rigorous impact evaluations are needed to fill the large knowledge gap in the field.

This paper is a product of the Impact Evaluation Team, Development Research Group. It is part of a larger effort by the World Bank to provide open access to its research and make a contribution to development policy discussions around the world. Policy Research Working Papers are also posted on the Web at http://econ.worldbank.org. The authors may be contacted at caiopiza@worldbank.org.

_The Policy Research Working Paper Series disseminates the findings of work in progress to encourage the exchange of ideas about development issues. An objective of the series is to get the findings out quickly, even if the presentations are less than fully polished. The papers carry the names of the authors and should be cited accordingly. The findings, interpretations, and conclusions expressed in this paper are entirely those of the authors. They do not necessarily represent the views of the International Bank for Reconstruction and Development/World Bank and its affiliated organizations, or those of the Executive Directors of the World Bank or the governments they represent._ Produced by the Research Support Team
**The Impact of Business Support Services for Small and Medium Enterprises on Firm Performance in Low- and Middle-Income Countries: A Meta-Analysis**

Tulio A. Cravo Inter-American Development Bank Labor Markets Division (SCL/LMK) tcravo@iadb.org Caio Piza World Bank Group Impact Evaluation Unit (DIME) caiopiza@worldbank.org Keywords: Private Sector Development, SME Development, Impact Evaluation, Firm Performance, and Meta-Analysis.

# JEL Classification: J21, J48, O10.

Acknowledgements: We thank Linnet Taylor, Lauro Gonzalez, Samer Abdenour, Anastasia de Santos, Vincenzo Salvucci, and Caroline Schimanski. We also thank participants in the USAID Microlinks seminar “Show Me the Data: Evidence & Experience on SMEs”. Excellent research assistance was provided by Ana Cristina Sierra, Isabel Musse and Isabela Furtado. Corresponding author: Caio Piza (caiopiza@worldbank.org).
# **Introduction**

Small and medium enterprises (SMEs) are responsible for the majority of employment generation in developed and developing countries (Ayyagari et al., 2007, 2011). Consequently, they play a central role in socioeconomic policies. In developing countries, business-support interventions are often based on the assumption that institutional constraints (or failures) impede SMEs from reaching their full potential to generate jobs, profits, economic growth, and alleviate poverty. Thus, the large financial resources allocated to the development of SME sectors by governments and development organizations is intended to address institutional constraints and allow SMEs to operate more efficiently, thus leading to productivity growth (Beck et al., 2005).

Development agencies provide a considerable amount of targeted assistance to SMEs in low- and middle-income country economies (Beck et al., 2006). For instance, the World Bank devoted $9.8 billion to SME projects during 2006-12 (IEG, 2013). For the same period, the International Finance Corporation (IFC) of the World Bank Group directed $25 billion to SMEs. However, there is limited evidence on the impact of SME support in the literature. This is due either to an insufficient number of studies employing convincing identification strategies to isolate the causal impact of the intervention under consideration or to limited information regarding the mechanism underlying such interventions.

There is a need to systematically review and synthesize the evidence to provide an account of the impact of different business-support programs on SMEs. This systematic review contributes to the public debate by providing an account of the effect of different types of direct support on firm performance. The evidence gathered and summarized is expected to help policy makers get a comprehensive overview of the literature and SME interventions that have been most effective. The review draws on economic theory to discuss the channels through which a particular intervention can affect firm-level outcomes and synthesizes evidence of existing interventions most frequently found in the literature: (i) matching grants, (ii) export promotion, (iii) innovation, (iv) training (technical assistance), (iv) cluster-based development, and (v) tax simplification policies. The aim is to synthesize the evidence of the impact of various interventions on different firm outcomes such as employment creation, exports, innovation, investment, and labor productivity, and firm performance indicators such as revenues and profits.

2
Understanding the mechanisms underlying each intervention is crucial if one is interested in designing SME interventions for different contexts. We try to provide as much information as possible on the potential causal chain of each intervention, given the information available in the literature included in this paper. This review also provides an account of the limitations related to the difficulties in the implementation of impact evaluations in the area of SME support and points out that this is, therefore, an area that requires further thorough research.

This work builds on previous related literature and systematic reviews that focused on specific sets of policies and included interventions that support micro-enterprise. For instance, McKenzie and Woodruff (2014) analyze business training interventions that include microenterprises and potential entrepreneurs. Similarly, Cho and Honorati (2014) focus on interventions promoting entrepreneurship among potential or current entrepreneurs. Finally, Grimm and Paffhausen (2015) provide a review more similar to this work by analyzing the impact of various types of SME support, but their work focuses only on employment outcomes and includes interventions with micro-entrepreneurs (for example, microfinance) and, in few cases, potential entrepreneurs.

Our research differs from previous ones in many ways. First, all evidence coming from studies with micro-enterprises are not covered in this review. We make this distinction because self-employed and micro-entrepreneurs targeted by microfinance interventions, for example, are thought to have a different nature compared to SMEs and are less likely to grow and create jobs with individual interventions. In fact, these enterprises are often ineligible for the public interventions covered in this review. Second, our review provides a thorough analysis of the impact of different types of SME support on various firm outcomes (not only on employment outcomes) and presents meta-analysis and meta-regression results disaggregated by type of intervention. Third, our results shed some light on the impact of matching grant interventions, one of the most popular interventions used by multilateral organizations such as the World Bank (Campos et al., 2012).

The findings suggest that overall SME business support has a positive impact on firm performance, employment creation, and labor productivity. When we look at interventions separately, matching grants stand out as effective in creating jobs and improving firm performance indicators. As will be discussed below, there is high variability in terms of number of studies per intervention and robustness of the evidence. The rest of the paper is organized as follows: Section 3
2 presents the logical framework associated with the interventions considered in this review. Section 3 describes inclusion criteria and search methods. Section 4 presents the search results and included studies. Section 5 presents the meta-regression methodology, section 6 shows the results, followed by the conclusion.

# **2. Logical Framework**

Various approaches are used to provide support services to SMEs. We identified the main among these approaches as relating to the following: formalization and the business environment,<sup>1</sup> volume exported (intensive margin), value chains and clusters, training and technical assistance, and finally, SME financing and innovation policies.

The literature on SME support can be divided into two distinct themes. The first considers indirect support that addresses constraints to SMEs accessing credit, while the second addresses the impact of direct business support on SMEs. In the first strand, many studies look at the impact of indirect types of public support for SMEs, such as tax simplification, which is intended to provide incentives for informal SMEs to formalize. The underlying assumption is that formal firms are less credit-constrained than their informal counterparts and therefore formalization is an effective way of helping entrepreneurs. Formalized firms are expected (assumed) to have higher economies of scale and, consequently, be more productive, demand a more skilled labor force, and have higher profits than informal firms. If informal firms are prevented from growing due to credit constraints, then reducing the cost of formalization should, in theory, indirectly give informal firms an opportunity to escape the informality-low productivity trap. Such interventions are an indirect form of public support, as they target all firms with annual revenues below a certain threshold. Moreover, all informal firms are incentivized to formalize through tax simplification. Those that formalize do not directly receive other forms of public support.<sup>2</sup> The second group of studies addresses the impact of direct business support on SMEs. These generally estimate the impact of a support program on SMEs within a specific sector in a

> 1 The Research Group at the World Bank conducted several experimental and quasi-experimental evaluations to investigate the impact of regulatory changes aimed at reducing bureaucratic barriers to SME formalization and growth. See Bruhn and McKenzie (2013) for a review.

> 2 In fact, there are interventions that are targeted at formal enterprises only, such as subsidized credit lines. Thus, it is possible that after formalizing, some firms may end up being served by different interventions.

4
given country, with the intervention based on the assumption that SMEs face specific constraints (for instance, a limited pool of skilled labor, limited innovation capability, and/or coordination failures). In this view, SMEs need public support to break through specific constraints and, in turn, improve their prospects for investment and productivity. A successful intervention may even generate spillover effects on firms that do not belong to the program’s target group. These may include firms in other sectors and/or informal firms in the same sector. This kind of support comes in the form of training programs and support for innovation or value chain and association strategies (for example, clusters), which are intended to address coordination failures. Notice that, unlike the indirect public support programs, the unit of intervention is the firm itself. Firms are directly targeted with programs that aim to help them shift from a low equilibrium (small size and scale) to a high equilibrium (bigger scale and dynamism).

As this review investigated the impact of a diverse array of interventions, we provide a theory of change for different types of interventions based on an initial search of the literature and provide the causal chain for each type of SME program analyzed.

Support to SMEs is generally related to the dual goals of productivity growth and employment generation. A general theory of change motivating SME support services is thus linked to the improvement or creation of institutions that allow SMEs to reach their full potential in growth and employment. Figure 1 below provides a general illustration of the simplified logical framework related to each type of intervention considered in this review. The description of the hypothesis entailed in each intervention model surveyed in this review is provided below.

5
<!-- Start of picture text --> Assumption: i<br>SMEs face various i<br>institutional<br>constraints and need<br>a ‘big push’to shift to —<br>a different<br>Direct: Intermediate Outcomes:<br>1. Matching grants 1. Lower costs (e.g.<br><!-- End of picture text --> <!-- Start of picture text --> 3. Training 2. Access to financial<br>4. Local production services<br>Systems 3. New skills, higher<br>5. Support to wages<br>innovation 4. Collective action<br>6. Export support 5. Spillover<br>(innovation)<br>6. Access to external<br>Public .<br>intervention markets Final1. Outcomes:Higher revenues and profits<br>2. Higher productivity<br><!-- End of picture text --> <!-- Start of picture text --> ; Intermediate Outcomes: °._ Employment generation<br>Indirect: 1. Formalisation<br>1. Taxsimplification 2. Access to credit<br>Assumption:<br>The institutional<br>environment is the<br>main barrierto the<br>success of SMEs<br><!-- End of picture text -->
training programs should contribute to worker employability and wages and to firm productivity (for example, through the adoption of more efficient management practices).

3) Interventions that support _local production systems_ (LPS). These are based on the idea that individual firms benefit from agglomeration externalities and coordination (for example, Schmitz, 1995). For instance, consider a project in a region specialized in a given sector providing incentives for firms to act collectively (such as training, joint purchases, or joint certifications). Economic theory suggests that formal firms might act together to capture collective externalities, experience mutual growth, and impact local economic performance. A successful project that allows firms to benefit from positive externalities generated by collective actions would affect outcomes such as employment and regional growth through: i) the establishment of collective agreements, and ii) specific outputs from collective action. The resulting causal chain is as follows: firms will organize around a common goal, enabling them to capture positive externalities from collective actions. Collective actions are expected to generate intermediate outputs that allow firms to achieve higher levels of productivity and employment and, in turn, positively impact regional economic performance. Interventions related to agglomeration economies also relate to value chains, networks, or clusters.<sup>3</sup> 4) Support for _innovation policies_ . These involve funding for improving processes (Lagace and Bourgault, 2003), and are intended to capture externalities stemming from innovations. Innovation programs aimed at SMEs might support innovation transfer, R&D programs, and certifications related to innovations (for example, process innovation and/or product differentiation). The rationale is that innovation will impact productivity and growth of firms, which contributes positively to regional and national growth.

5) Public intervention supporting _access to external markets_ . Such interventions seek to tackle information asymmetries that prevent firms from accessing external markets and involve providing training and counselling. The identification and adaptation to external markets generates exports that may lead to increased production, which, in turn, are thought to impact firm profits and employment creation.

> 3 Like the papers included in this review, we do not try to provide a specific and precise definition of local agglomeration. For more about the difficulties related to the concept and definition of spatial agglomerations, please see Altenburg and Meyer-Stamer, (1999) and Martin and Sunley (2003).

7
6) _Tax simplification_ . These initiatives are a form of indirect business support to SMEs and are aimed at improving firm performance through formalization. Economic theory suggests that formal firms grow by accessing credit markets and by taking advantage of economies of scale. A tax simplification program could affect outcomes such as employment and profits through two intermediate outcomes: 1) formalization rates and 2) access to credit. The causal chain could be simplified as follows: the necessary conditions for a tax simplification program shifts informal entrepreneurs from an equilibrium characterized by low productivity and profits to another where they face fewer constraints to growth (as a result of formalization). Plenty of studies concentrate only on final outcomes and thus shed little light on the mechanisms associated with tax simplification/formalization (and consequently offer little policy guidance). The underlying assumption is that formal firms are less credit-constrained than their informal counterparts and, therefore, formalization is an effective way to help entrepreneurs. Indirect support to SMEs may include policies regarding business registration, property registration, and regulatory frameworks (Fajnzylber et al., 2011; Monteiro and Assunção, 2012; McKenzie, 2013).

The various result chains shown in Figure 1 are thus useful in providing the rationale behind the types of interventions considered in this review.

# **3. Inclusion Criteria and Search Methods**

This review focuses on studies that evaluate policies aimed at supporting SMEs in LMICs (as defined by the World Bank). The focus on LMICs is justified, firstly, because private firms in these countries tend to be more labor intensive and less innovative and, consequently, are the main employers of a large proportion of the labor force (for example, Acz and Amoros, 2008; Cravo et al., 2012). Secondly, restricting the scope to LMICs helps identify the binding constraints that SMEs might face in similar institutional contexts.

A common definition of SMEs does not exist. This review mainly uses the most common criteria used to classify SMEs based on employment information. The cut-off used to define SMEs is 250 employees as Beck et al. (2005), Ayyagari et al. (2007), Cravo et al. (2012), Kushnir et al.

8
(2010).<sup>4</sup> We also included studies that do not consider number of employees but use annual revenue (based on national classifications) instead to classify SMEs. Importantly, as mentioned above, interventions supporting entrepreneurship and the creation of micro-enterprises (for instance, microfinance) are not considered for this research. These businesses, especially in LMICs, comprise less productive or informal enterprises with only a few employees at the fringes of markets. This is a major difference in our review when compared to Grimm and Paffhausen (2015) who included studies focused on self-employed and microfinance. Further, these enterprises are often ineligible for the public interventions covered in this review.

To examine the evidence on the effect of SME support services on firms, this review focused on quantitative analysis and included only studies that used experimental (randomized controlled trials, or RCTs) and quasi-experimental methods – such as regression discontinuity design (RDD), instrumental variables (IV), difference-in-differences (DID), matching on covariates, propensity-score matching (PSM), and any other studies that purported to control for selection bias (for example, Heckman two-step estimator). Experimental and quasi-experimental methods are regarded as good tools when the main objective is to estimate the causal impact of an intervention or policy (for example, see Duflo et al. 2008). When an intervention is carefully designed or the identification strategy of an observational study convincing enough, the findings on the impact of the program or intervention are said to have internal validity. That is, one can claim that the difference in the outcomes between treatment and control groups was caused by the intervention. This review only considered those studies that assessed the impact of an intervention comparing the treatment (or eligible) and the control (or comparison) groups. Moreover, studies using matching methods needed to clearly state the eligibility criteria of the program to make the case that the problem of selection bias was (mostly) due to observed characteristics.

Importantly, as described in the previous section, this review includes studies that considered the impact of six different types of business-support services based on firm performance. In addition, our study is more complete as it examines different firm-level outcomes and does not restrict the analysis to employment outcomes as in Grimm and Paffhausen (2014).<sup>5</sup> Our review covers studies that looked at both intermediate (or secondary) outcomes (such as access

> 4 Further, the European Union and the World Bank (see, for instance, the Enterprise Survey website <u>www.enterprisesurveys.org) adopt 250 employees as a cut-off to classify SMEs.</u>

> 5 Though the literature recommends that synthesis is informed by the theory of change embedded in the design of an intervention (see Waddington et al., 2012b), our focus extends beyond the outcomes directly anticipated by an intervention to also include unanticipated outcomes.

9
to credit, training, formalization, and access to external markets) and final (or primary) outcomes (such as profits, employment generation, and productivity). To be included in the review the study had to report estimates to at least one final outcome.<sup>6</sup> Studies that reported estimates for intermediary outcomes only were excluded. This review looked for context-specific variables that can help explain either the failure or success of an intervention to understand the causal chain of each intervention.

# **_Search Methods_**

Following the setting up of the inclusion criteria, different search strategies were devised to identify studies to be included in the review. The generalized search strategy covered a comprehensive set of published and unpublished sources. We prioritized electronic searches since, regarding interventions of interest; it was most likely that sources available electronically were reported in formal literature on SMEs or in the ‘grey literature’ from national and international organizations.

The first stage of the review involved a search for all published and unpublished studies likely to be relevant to our objectives. To be included, the studies had to: i) report on SME support interventions of the kind detailed in the section on interventions; ii) focus on LMICs, as defined by the World Bank; and, iii) have occurred after 2000, since the review would cover studies that used impact evaluation techniques that evolved since that year.

Given the variety of interventions covered in this research, reference ‘snowballing’ was an effective strategy to begin our search (Hammerstrøm et al., 2009; cited in Waddington et al., 2012). Reference snowballing consists of using existing reviews, papers, and reports to identify the set of studies to be reviewed. Our search strategy, therefore, also drew on a first set of important studies identified in an initial screening. We then conducted the electronic search that is described in detail in appendix A and Piza et al. (2016).

> 6 The selected studies reported on at least one impact relating to firm outcomes, either intermediary or final. For the purposes of this review, we defined firm performance impacts as referring to objective indicators such as revenues, profits, job creation, innovation, formalization, number of workers trained, and access to credit. Only factual/objective measures of firm performance impacts are included: subjective measures on beliefs and perceptions are excluded.

10
# **4. Search Results and Included Studies**

# 4.1. Search Results

The initial electronic search returned 9,475 studies, which was reduced to 5,785 after dropping of duplicates. The final list of studies was examined with all filters outlined above, which assessed the impact of an SME intervention using rigorous evaluation methods. With that in mind, abstracts of all 5,785 studies were read. It was noted that the great majority either did not use quantitative methods to assess the impact of an intervention nor used a rigorous method to address selection problems or looked at interventions targeting micro-entrepreneurs.

Three researchers, working independently, were involved in applying the selection criteria. They read the abstracts and drew up a list of 63 papers that passed all filters. The list dropped to 42 after excluding 21 studies that only covered micro-enterprises. The papers were then classified according to the methods used: quasi-experimental and experimental methods respectively.

The 42 studies where thoroughly examined to decide whether they should be included in the review. We excluded six studies that looked exclusively at intermediate outcomes – such as formalization rates and numbers of new firms – and different versions of the same study. We also excluded 13 studies that did not use rigorous evaluation methods to address causality. The snowballing strategy added 17 studies and generated a final list of 40 studies (23 from the search of online platforms and 17 from snowballing). A further four studies were dropped because we were unable to compute a standardized effect size and/or their standard errors. To compare effect sizes across studies, we used two standardized measures reported in section 5.1 and described in detail in appendix B.

The empirical analysis, therefore, included 36 studies and 72 effect size (ES) per intervention-outcome study. The large number of ES is because a few studies tested the impact of several interventions together and then separately on the same outcomes and some randomized controlled trials tested the effect of more than one treatment arm.

11
# 4.2. Included Studies

Figure 2 shows the cumulative number of studies produced between 2003 and 2014. Between 2003 and 2010, only 16 studies used experimental or quasi-experimental techniques to assess the impact of different business support to SMEs. Between 2011 and 2014 that number more than doubled.

**Figure 2 – Cumulative Number of Studies Per Year**

<!-- Start of picture text --> 40<br>38<br>35<br>29<br>16<br>11<br>10<br>6<br>5<br>3<br>1<br>2003 2005 2006 2007 2008 2009 2010 2011 2012 2013 2014<br><!-- End of picture text --> Source: Own elaboration Figure 3 shows that the evidence from 18 countries, most of which are in the Latin American region. As noted in Grimm and Paffhausen (2015), this could be because countries in this region have many experiences with active labor market policies over the past two decades.

12
# **Figure 3 – Number of Studies Per Country**

Source: Own elaboration The review of the papers included in the evaluation allowed us to analyze the effect of the interventions on a comprehensive list of outcomes. The final outcomes extracted from the papers reviewed are: i) employment creation; ii) labor productivity; and iii) firm performance. The following measures were extracted from the papers reviewed for intermediary outcomes: i) access to credit; ii) exports; iii) formalization rates; iv) innovation; v) investment; and vi) survival rates.

For firm performance, we grouped various outcomes such as profits, revenues, sales, added value, stock of assets, return on assets, gross production, and firm productivity (measured as total factor productivity). For employment, we grouped paid workers, new workers, workers recruited, and employment rates. Innovation encompasses all types of investments in research and development (R&D), new products, and patents. Our measure of labor productivity grouped studies that reported sales per worker, profit per worker, revenue per worker, and R&D per worker. Figure 4 reports the percentage distribution of reported outcomes (72 in total). Four outcomes stand out: firm performance (27.8 percent), employment (20.1 percent), exports (15.3 percent), labor productivity (11.1 percent), and investment and innovation (8.3 percent).

13
# <mark>en</mark>
overall estimate and their p-values to assess statistical significance. To provide a more robust set of results, meta-regressions are used to analyze the impact of SME support programs on firm outcomes controlling for moderator factors.

# **5.1 Computing Effect Sizes**

Most studies included in this paper use quasi-experimental methods to estimate the causal effect of a program. The majority of papers estimate the average treatment effect on the treated (ATT), but few estimate the local average treatment effect (LATE) instead.

For our meta-analyses, the unit of analysis was the study.<sup>7</sup> Nonetheless, several studies performed more than one estimate for the same outcomes. For example, in some cases, studies report on different interventions and in others, different specifications are tested for the same intervention. In any case, there was a need to synthesise several estimates for the same intervention (for example, matching grant) and outcomes (for example, employment). When a study covered more than one treatment (for example, matching grants and technical assistance) and provided estimates for each treatment separately and for ‘whatever’ treatment without distinguishing between the two treatments, we opted to use only the latter estimate to compute overall effect size when all different interventions were pooled.<sup>8</sup> In this case, the treatment dummy is defined as one if a firm is supported by ‘any program’ (in the example, either matching grants or technical assistance) and zero if not (as in Hong Tan, 2011; López-Acevedo et al., 2011).

When such ‘synthetic effect’ is not provided, we determined it by taking a simple average of the ES across different interventions per outcome per study (Lipsey and Wilson, 2001). In such cases, the variance of different effect sizes was computed assuming zero covariance because in most cases overlap was limited. That is, firms either participated in one program or

> 7 As discussed in Duvendack et al. (2012), there is not a consensus of whether meta-analysis should be performed for quasiexperimental studies. In this review we decided to use meta-analysis to have the ‘big picture’ of the impact of interventions aimed at SMEs. However, in face of the challenges in practice and decisions made, we argue that these results should be treated with care.

> 8 Alternatively, we could have computed a weighted average of two separated coefficients.

15
another.<sup>9</sup> Averaging out across standardised ES provided in the same study was necessary to generate one overall ES per outcome per study so we could carry out meta-analysis pooling together different business-support programs.

We also performed subgroup analyses, looking at some interventions separately. For instance, our review reports on a relatively high number of studies looking at the effect of matching grants on firm outcomes. In cases where the same study tested the impact of more than one intervention (for example, matching grants and technical assistance), we first averaged the ES for matching grants and technical assistance separately and then took a simple average to obtain an overall ES per outcome per study. As before, this was to estimate an overall standardized ES across different intervention; and again we computed the variance assuming covariance between effect sizes as zero.<sup>10</sup> When sample sizes and treatment effects for subgroups are available, we computed summary effects as a weighted average of the effects’ sizes. As before, we also computed the variance by assuming covariance between the ES equals zero because this seems to be a plausible assumption for cases where overlap between subgroups is non-existent or small, that is, where the ES are plausibly independent.

In sum, we provide synthesised ES for three primary outcomes: (1) firm performance; (2) employment; and (3) labour productivity. For four secondary outcomes, (a) exports, (b) investment, (c) innovation, and (d) formalization rates, we show the forest plots with individual estimates in the appendix since we did not systematically review studies looking specifically at those outcomes. The effect sizes used to construct forest plots for the initial analysis are subsequently used in the meta-regression estimations.

After obtaining the effect sizes and their respective SE per outcome per study, we computed forest plots for an initial visualization of the results.

> 9 Since variance of (a+b) = var(a) + var(b) – 2 Cov(a,b), assuming Cov(a,b) = 0 is a conservative assumption as it implies lower precision of overall effects unless the covariance is negative. On average, we expect the covariance across studies to be close to zero. We also believe this is a reasonable assumption because, according to these studies, the number of firms taking up different treatments is not high. Given the restricted overlap between different treatments, we do not believe there is reason to worry about high correlation between firms participating in different interventions. It is important to clarify that by doing this we are not averaging across outcomes, but instead, across different ES for a given outcome.

> 10 In other words, we did not combine estimates obtained for firms receiving matching grants only with estimates for firms receiving package of interventions (for example, matching grants and technical assistance).

16
# **6. Results**

This section provides an overview of the overall average effect of business-support programs to SMEs. We start by aggregating all interventions and providing evidence for single interventions when sample size (number of studies) allows. We use forest plots and random effect estimates to compute the average standardized effect size and use I-squared and tau-squared statistics to compute variability of our main findings.<sup>11</sup> The results are summarized for the final (or primary) outcomes of employment, productivity, and firm performance.

# **6.1 Forest Plot Analysis**

Our review found 18 ES related to firm performance across different interventions as illustrated in panel A of the forest plot (figure 5).<sup>12</sup> The forest plot reports the standardized ES (SMD) of each study and the overall average across interventions. The interventions included in this figure consider different group of firms (for example, sector) and aim to tackle different market failures. Nevertheless, providing an overall picture of the interventions covered in the review can still be relevant for policy making.<sup>13</sup> On average, interventions aimed at improving firm performance had a positive and significant effect of 0.13 standard deviations. Interestingly, the heterogeneity between studies is relatively small. The tau-squared is very low (0.0196). As indicated by the statistic I-squared (92.1 percent), there is an indication of high heterogeneity across studies. This measure captures the degree of inconsistency in the studies’ results (Higgins et al., 2003).

Since our review included seven ES for studies that examined the impact of matching grants programs, our data allows us to look at the effect of these two interventions on firm

> 11 We report forest plot and heterogeneity measures, such as the Chi-squared test for heterogeneity (which captures within-study variance), the I-squared statistic, which we interpret as the proportion of total variance across the observed effects explained by between-study variance, and τ^2 (tau-squared), an estimate for the variance of the ‘true effect size’ (see Borenstein et al., 2009). Borenstein et al. (2009, p.118) argue that “I-squared is a descriptive statistic and not an estimate for any underlying quantity.” 12 Figure 5 reports forest plots dropping studies with ES that are outliers. The results with the full set of observations are similar (see Piza et al., 2016).

> 13 The decision to report overall effect for different interventions was also made, for instance, in a systematic review that covered the impact of interventions aimed at improving children’s enrollment in primary and secondary schools. See Petrosino et al., 2012.

17
performance in isolation. Panel B of **figure 5** shows that the effect of MG on firm performance equals 0.15 and is similar to that obtained with all interventions pooled together.

The number of ES for employment outcome is 13 (see panel C, **figure 5** ). Although most of the evidence comes from Latin America, the figure suggests that different types of business support for SMEs help create jobs in almost all the countries considered. On average, programs targeted at SMEs tend to help with employment creation. The overall effect is equal to 0.15 standard deviations and statistically significant. Despite the smaller number of cases, the tausquared statistic points to a between-study variance of 0.081; that is, the between-study variance accounts for more than 50 percent of the pooled effect size (0.08/0.15). The high value of I-squared statistic (99.2 percent), though, indicates a high true between-study variability. This result is consistent with the view that SMEs are an important source of job creation. When we look at the effect of matching grants on employment (panel D), the results are similar with a positive effect size of 0.14 SD. Nevertheless, the reduction in the number of studies leads to higher variability between the point estimates as captured by the tau-squared (0.133) and I-squared statistics (99.4 percent).

The number of ES results for labor productivity is seven. The evidence comes almost exclusively from countries in Latin America (see panel E). The overall effect size is 0.11, indicating that SME support might affect productivity. The overall variance is relatively low as the I-squared statistic indicates that 88.7 percent of the total variance is explained by betweenstudies variability and the tau-squared is low (0.0117). When we look only at the effect of matching grants, we find a small effect that is not statistically different from zero (-0.02 SD with a 95 CI of (-0.15, 0.10)) – see figure 5, panel F.

The initial indication of a positive impact of SME support on firm performance is interesting and can have at least two possible interpretations. First, it can be argued that business support of any sort works as subsidies (‘free money’) that end up favoring firms that would actually be able to carry on without any injection of public resources, that is, a _picking the winners_ argument. On the other hand, one could take this result as an indication that SME interventions of any sort are key to SMEs needing a ‘nudge’ to increase performance (or survive). To shed light on these two competing views, we looked at the effect of MG on secondary outcomes, such as investment. There seem to be some positive effects on investment, as shown in figure D.9 in the 18
appendix. In the meta-regression analysis, we also approached this issue indirectly by looking at whether firm size is associated with the final outcomes.

# Figure 5 - Forest plot - final outcomes

|Author|year|country|**ES (95% CI)**|**%**<br>**Weight**|
|---|---|---|---|---|
|<br>Rijkers et al|2010|<br>Ethiopia|<br>**-0.13 (-0.39, 0.12)**|**4.10**|
|Aivazian & Santor|2008|Sri Lanka|**-0.10 (-0.32, 0.13)**|**4.55**|
|Karlan et al|2014|Ghana|**-0.06 (-0.30, 0.18)**|**4.25**|
|Tan & Lopez-Acevedo|2005|Mexico|**-0.02 (-0.13, 0.09)**|**6.55**|
|Crespi et al|2011|Colombia|**0.02 (0.01, 0.04)**|**7.62**|
|Atkin et al|2014|Egypt|**0.05 (-0.22, 0.32)**|**3.85**|
|Rand & Torm|2011|Vietnam|**0.09 (-0.02, 0.20)**|**6.50**|
|Bruhn et al|2012|Mexico|**0.10 (-0.10, 0.30)**|**5.00**|
|Gourdon et al|2011|Tunisia|**0.11 (-0.12, 0.34)**|**4.50**|
|Oh et al|2008|Korea|**0.17 (0.12, 0.21)**|**7.46**|
|McKenzie & Sakho|2009|Bolivia|**0.17 (-0.04, 0.37)**|**4.90**|
|Arraiz et al|2012|Chile|**0.18 (0.07, 0.28)**|**6.62**|
|Sekkat|2011|Marocco|**0.20 (0.02, 0.39)**|**5.17**|
|Lopez-Acevedo & Tinajero|2011|Mexico|**0.21 (0.13, 0.29)**|**7.08**|
|De Mel et al|2011|Sri Lanka|**0.22 (-0.05, 0.49)**|**3.83**|
|Cassano et al|2013|RUS/BGR/UKR/GEO|**0.26 (0.14, 0.37)**<br>|**6.48**|
|Benavente and Crespi|2003|Chile|**0.28 (0.03, 0.54)**|**4.10**|
|Fajnzylber et al.|2012|Brazil|**0.34 (0.30, 0.39)**|**7.43**|
|Overall (I-squared = 92.0%, p =|0.000)||**0.13 (0.05, 0.20)**|**100.00**|
|NOTE: Weights are from rando|m effects anal<br>Panel|ysis<br>**0**<br>-.538<br>.<br>A - All interventions: Firm Perform|538<br>ance||

|Author|year|country<br>**ES (95% CI)**|**Weight**<br>**%**|
|---|---|---|---|
|Hong Tan|2011|Chile<br>**-0.24 (-0.28, -0.19)**|**12.17**|
|Rijkers et al|2010|Ethiopia<br>**-0.13 (-0.39, 0.12)**|**9.73**|
|Karlan et al|2014|Ghana<br>**-0.02 (-0.27, 0.22)**|**9.89**|
|Rand and Torm|2011|Vietnam<br>**0.09 (-0.02, 0.20)**|**11.66**|
|Oh et al|2008|Korea<br>**0.17 (0.12, 0.21)**|**12.19**|
|Arraiz et al|2012|Chile<br>**0.18 (0.07, 0.28)**|**11.73**|
|Cassano et al|2013|RUS/BGR/UKR/GEO<br>**0.26 (0.14, 0.37)**|**11.65**|
|Benavente & Crespi|2003|Chile<br>**0.28 (0.03, 0.54)**|**9.73**|
|Duque & Munoz|2011|Colombia<br>**0.56 (0.42, 0.71)**|**11.26**|
|Overall (I-squared = 9|6.5%, p|= 0.000)<br>**0.13 (-0.04, 0.30)**|**100.00**|
|NOTE: Weights are fr|om rando|m effects analysis||
|||**0**<br>-.714<br>.714||
||P|anel B - Matching Grants: Firm Performance||

19
# Figure 5 - Forest plot - final outcomes (continued)

|Author|year|country|**ES (95% CI)**|**Weight**<br>**%**|
|---|---|---|---|---|
|Karlan et al|2014|Ghana|**-0.09 (-0.33, 0.15)**|**4.88**|
|Crespi et al|2011|Colombia|**0.02 (0.00, 0.04)**|**10.56**|
|Hong Tan|2011|Chile|**0.05 (-0.00, 0.10)**|**10.14**|
|Machado et al|2011|Brazil|**0.09 (0.03, 0.15)**|**9.88**|
|Bruhn et al|2012|Mexico|**0.12 (-0.08, 0.32)**|**5.96**|
|Arraiz et al|2012|Chile|**0.13 (0.02, 0.24)**|**8.65**|
|Oh et al|2008|Korea|**0.14 (0.10, 0.19)**|**10.23**|
|De Mel et al|2011|Sri Lanka|**0.16 (-0.11, 0.44)**|**4.31**|
|Lopez-Acevedo & Tinajero|2011|Mexico|**0.18 (0.10, 0.25)**|**9.50**|
|Kaplan et al|2011|Mexico|**0.22 (-0.21, 0.65)**|**2.28**|
|Castillo et al|2010|Argentina|**0.32 (0.27, 0.38)**|**10.02**|
|Gourdon et al|2011|Tunisia|**0.33 (0.10, 0.56)**|**5.20**|
|Cassano et al|2013|RUS/BGR/UKR/GEO|**0.35 (0.23, 0.46)**|**8.39**|
|Overall (I-squared = 92.8%, p =|0.000)||**0.15 (0.08, 0.22)**|**100.00**|
|NOTE: Weights are from rando|m effects ana<br>Pane|lysis<br>**0**<br>-.649<br>.6<br>l C - All interventions: Employm|49<br>ent||

Panel C - All interventions: Employment **%** Author year country **ES (95% CI) Weight** Karlan et al 2014 Ghana **-0.19 (-0.43, 0.06) 8.81** Hong Tan 2011 Chile **-0.00 (-0.05, 0.04) 15.92** Machado et al 2011 Brazil **0.09 (0.03, 0.15) 15.61** Arraiz et al 2012 Chile **0.13 (0.02, 0.24) 14.10** Oh et al 2008 Korea **0.14 (0.10, 0.19) 16.02** Castillo et al 2010 Argentina **0.32 (0.27, 0.38) 15.77** Cassano et al 2013 RUS/BGR/UKR/GEO **0.35 (0.23, 0.46) 13.77** Overall (I-squared = 93.8%, p = 0.000) **0.14 (0.03, 0.24) 100.00** NOTE: Weights are from random effects analysis -.464 **0** .464 Panel D - Matching Grants: Employment Creation 20
# Figure 5 - Forest plot - final outcomes (continued)

|Author|year|country|**ES (95% CI)**|**%**<br>**Weight**|
|---|---|---|---|---|
|Chudnovsky et al|2006|Argentina|**-0.07 (-0.28, 0.15)**|**2.41**|
|Castillo et al|2010|Argentina|**0.07 (0.01, 0.12)**|**21.68**|
|Sanguinetti|2005|Argentina|**0.08 (-0.17, 0.32)**|**1.86**|
|Lopez-Acevedo & Tinajero|2011|Mexico|**0.10 (0.03, 0.18)**|**14.13**|
|Oh et al|2008|Korea|**0.14 (0.09, 0.18)**|**27.26**|
|Arraiz et al|2012|Chile|**0.15 (0.04, 0.25)**|**8.43**|
|Hong Tan|2011|Chile|**0.15 (0.10, 0.20)**|**24.23**|
|NOTE: Weights are from ran<br>Overall (I-squared = 31.3%,|dom effects<br>p = 0.189)|analysis|**0.11 (0.08, 0.15)**|**100.00**|

-.325 **0** .325 Panel E - All interventions: Productivity

|Author|year|country|**ES (95% CI)**|**Weight**<br>**%**|
|---|---|---|---|---|
|Hong Tan|2011|Chile|**-0.07 (-0.12, -0.02)**|**23.14**|
|Chudnovsky et al|2006|Argentina|**-0.07 (-0.28, 0.15)**|**11.48**|
|Castillo et al|2010|Argentina|**0.07 (0.01, 0.12)**|**22.84**|
|Oh et al|2008|Korea|**0.14 (0.09, 0.18)**|**23.43**|
|Arraiz et al|2012|Chile|**0.15 (0.04, 0.25)**|**19.11**|
|Overall (I-squared =|90.6%, p|= 0.000)|**0.05 (-0.05, 0.15)**|**100.00**|
|NOTE: Weights are|from rand|om effects analysis<br>**0**<br>-.283<br>.<br>Panel F - Matching Grants: Pr|283<br>oductivity||

21
# **_Meta-Regression_**

The forest plots presented earlier provide a useful preliminary discussion about the effect size of SME-support programs. However, forest plots are not able to control for moderator factors (for example, size of firms and regional characteristics and risk of bias of studies). Metaregressions are estimated to provide a better account of effect size related to SME-support programs.

The meta-regression analysis is performed for the pooled sample of interventions and for matching grants separately. For matching grants we are able to control – separately due to sample constraints – for another three secondary outcomes: investment, access to export, and innovation. The overall effect was estimated using a random effects (RE) model. A RE model assumes there might be different ES underlying different studies and interventions and that the total variance for these should account for between-studies variance (see Borenstein et al., 2009). We also report the confidence interval for each overall estimate and its p-value to assess statistical significance. The baseline framework is as follows: where �� is the outcome, �� includes the type of intervention and �� is the error term. Extensions of the baseline model include four additional moderator factors; Latin America and Africa variables, firm size, and risk of bias indicator created based on a careful risk of bias assessment (see appendix D). The meta-regressions are estimated for final and intermediary outcomes.

# **_Primary Outcomes_**

Table 1 shows the coefficients for the meta-regression. The first row shows the random effects estimates without controlling for any moderator factor. The coefficients are identical to those reported in the forest plot once outliers are excluded. These estimates correspond to the overall mean effect as showed in the forest plots.

We then estimate meta-regression controlling for each moderator factor in separated regressions. We had to estimate each regression one-by-one due to insufficient sample size. We report the coefficient for the constant (RE when the dummy variable takes the value of zero) and
# the coefficient of the moderator variable in all cases.

Table 1 – Meta-Regression for Primary <u>Outcomes</u> _<u>(Excluding Outliers)</u>_

||Firm|Employment|Labor|
|---|---|---|---|
||Performance|Creation|Productivity|
|RE estimate -- no controls|0.13***|0.15***|0.11***|
|p-value|0.000|0.001|0.001|
|N|19|13||
|**_Moderator variables (Control variables)_**||||
|Constant|0.10**|0.19***|0.14**|
|p-value|0.036|0.01|0.014|
|_LAC fixed effect (1 if LAC; 0 otherwise)_|0.057|-0.06|-0.03|
|p-value|0.35|0.43|0.48|
|N|19|13|7|
|Constant|0.15***|0.15***|Na|
|p-value|0.000|0.002||
|_Africa fixed effect (1 if Africa; 0 otherwise)_|-0.10|-0.03|Na|
|p-value|0.18|0.82||
|N|19|13||
|Constant|0.16***|0.21***|0.13|
|p-value|0.000|0.004|0.11|
|_Firm size (continuous variable)_|-0.001*|-0.001*|-0.0003|
|p-value|0.06|0.15|0.70|
|N|19|13|7|
|Constant|0.09**|0.074|0.11**|
|p-value|0.047|0.21|0.027|
|_Risk of bias (1 for moderate or high RoB; 0 for low_||||
|_RoB)_|0.08|0.11|0.00|
|p-value|0.17|0.12|0.99|
|N|19|13|7|
|Constant|0.14***|0.16***|Na|
|p-value|0.000|0.002||
|_Method (1 if RCTs; 0 if QE)_|-0.07|-0.08|Na|
|p-value|0.42|0.42||
|N|19|13||

Note: ***, **, * Statistically significant at 1, 5, and 10 percent respectively.

Given the small sample of studies, these estimates are underpowered. The lack of statistical significance should not mean that these moderator factors are unimportant. The magnitude of the effect size and its sign can be informative but should be interpreted with caution in such a context.

First, the coefficient of the dummy variable for LAC is positive but statistically 23
insignificant. The estimate indicates that business-support services implemented in LAC is associated, on average, with higher effects on firm performance. However, for the other two outcomes, we observe the opposite, that business-support services implemented in LAC are associated, on average, with lower effects on employment creation and labor productivity, by 0.06 of a SD and to 0.03 of a SD respectively. As before, the estimates are not significant in statistical terms. We have insufficient data to explore this issue further, but it could be that business support to SMEs in LAC leads to more capital-intensive technology and therefore is less likely to create jobs.

The estimate for the ‘Africa’ dummy indicates that SME support programs in Africa are associated with a lower pooled effect on firm performance, but is only marginally associated with lower effect on employment creation. The size of firms may play a role in the main findings. As can be seen in the table, the random effects estimate increases in all three cases once we control for firm size, suggesting that larger firms are associated with larger impacts. The relationship might not be linear though.<sup>14</sup> Figure E.1 in the appendix shows the histogram for this variable. The figure highlights that most of the firms assessed in the studies covered by this review have fewer than 100 employees. A high percentage (25 percent) has no more than 10 employees (first bar). For studies covering African countries, the median size of firms is 93 and the mean is 83. This indicates that there is a larger proportion of small firms studied in Africa, given the left-skewed distribution.

Table 1 shows the random effects estimates once risk of bias is controlled for. Because the dummy risk of bias takes the value of 1 for studies with a high risk of bias, the significant reduction in the magnitude of the effects indicates that high-risk studies tend to show more positive results on firm performance than studies with low or moderate levels of bias. The same holds for employment creation, but not for labor productivity. In fact, once a dummy for risk of bias is added to the model, the effect on employment turns statistically insignificant. One could interpret these results as a signal that the most rigorous studies have not found effects of business interventions on these firms’ performance and employment creation. Therefore, with so few good studies available, any conclusion regarding the effects of such interventions should be

> 14 We tested a quadratic specification for the variable size; the coefficients for the quadratic term are very often negative, suggesting a concave relationship between firm size and firm performance. Because number of studies is relatively small, the estimates are imprecisely estimated and are available upon request.

24
interpreted with caution.

Finally, the coefficient of the dummy variable that informs the method used (1 for RCT and zero for quasi-experimental methods) suggests that the RCTs included in this review were less likely to find positive effects on firm performance and employment creation. We believe that this might be in part due to the scales of the programs evaluated. Studies using quasiexperimental methods usually rely on administrative data sets with thousands of observations whereas RCTs might test programs in their pilot stages.

Table 2 replicates the exercise for MG interventions only. The results for firm performance are qualitatively similar to those presented in table 1 and few estimates stand out. First, the coefficient of the dummy ‘Africa’ is large and negative in the first column, suggesting that MG programs in Africa are associated with worse performance of firms. On the other hand, the coefficient for Africa region is positive and relatively large for employment creation. This suggests that MG interventions in African countries were more likely to create jobs. This is consistent with the hypothesis that African firms’ production functions may be more labor intensive (than LAC, for instance) and that they likely work at relatively low scales, hence the scope to grow through addition of labor inputs.

25
Table 2 – Meta-Regression for Primary Outcomes _Matching_ _<u>Grants (Exclude Outliers)</u>_

||Firm Performance|EmploymentCreation|Labor Productivity|
|---|---|---|---|
|RE Estimate -- No Controls|0.15**|0.13*|0.052|
|p-value|0.012|0.083|0.33|
|N|7|7|5|
|**_Moderator Variables_**<br>||||
|**_(Control Variables)_**||||
|Constant|0.11*|0.13|0.14|
|p-value|0.095|0.305|0.244|
|_LAC Fixed Effect (1 if LAC; 0_||||
|_otherwise)_|0.10|0.13|0.14|
|p-value|0.40|0.305|0.244|
|N|7|7|5|
|Constant|0.17***|0.17**|Na|
|p-value|0.000|0.029|Na|
|_Africa Fixed Effect (1 if_||||
|_Africa; 0 otherwise)_|-0.27**|0.17**|Na|
|p-value|0.03|0.029|Na|
|N|7|7|Na|
|Constant|0.17*|0.27*|0.24|
|p-value|0.084|0.053|0.113|
|_Firm Size (Continuous_||||
|_Variable)_|-0.001|0.27*|0.24|
|p-value|0.37|0.053|0.113|
|N|7|7|5|
|Constant|0.15|0.015|0.068|
|p-value|0.131|0.33|0.501|
|_Risk of Bias (1 for moderate_||||
|_and high risk of bias; 0 for_||||
|_low)_|-0.01|0.015|0.068|
|p-value|0.94|0.33|0.501|
|N|7|7|5|
|Constant|0.16***|0.20**|Na|
|p-value|0.002|0.018|Na|
|_Method (1 if RCTs; 0 if QE)_|-0.23|0.20**|Na|
|p-value|0.27|0.018|Na|
|N|7|7|Na|
|Constant|0.15**|0.16*|0.10*|
|p-value|0.012|0.074|0.047|
|_Export (Continuous Variable)_|2.23**|2.86|-2.85**|
|p-value|0.02|0.11|0.012|
|N|7|7|5|

26
|_Cont. Table 2_||||
|---|---|---|---|
|Constant|0.06|0.13|0.06|
|p-value|0.48|0.16|0.37|
|_Innovation (Continuous_<br>_Variable)_|6.32|8.23|-1.85|
|p-value|0.15|0.23|0.59|
|N|7|7|5|
|Constant|0.08|0.17**|0.025|
|p-value|0.36|0.027|0.67|
|_Investment (Continuous_<br>_Variable)_|-0.92|-2.99***|8.00|
|p-value|0.35|0.01|0.52|
|N|7|7|5|

Note: ***, **, * Statistically significant at 1, 5, and 10 percent respectively.

The coefficient for size of firms is positive and large in all three columns, though not statistically significant for labor productivity. The result in the second column could be picking up a mechanical effect since firm sizes are measured as numbers of employees.

MG programs that aimed at improving firm capacities to export and innovate showed positive effects on firm performance and employment creation, but negative on labor productivity. This result is puzzling and we interpret it as an indication that firms targeted by the type of interventions covered in this review were likely facing some constraint to increase output beyond the variable costs associated with extra hired labor. This could also reflect some distortion in case an intervention somehow incentivized firms to create jobs (for example, unpaid jobs through employment of family members) through different forms of subsidies (for example, wage subsidy).

Finally, the coefficient for the variable ‘investment’ was negative for firm performance and employment creation. Our interpretation is that the investments made by these firms might have been toward addition of capital goods.

In a nutshell, these findings suggest that matching grants serve different firm composition and business purposes. Export-oriented firms, for example, need to become more efficient to be able to compete in the external market while labor-intensive firms may use matching grants to hire extra labor.

27
# **_Study Limitations_**

Most of the studies covered in this review employ quasi-experimental designs that rely on assumptions that may fail at controlling for all sources of confounders. The process of elaboration of this review confirmed a point made by Baird et al. (2013) that very few economic papers report the exact information necessary to perform ES calculations. So assumptions had to be made. In addition, to synthesize the ES across different studies, we made a considerable simplification in averaging SMD obtained through estimation of different parameters – such as intention to treat (ITT) often reported in RCTs, average treatment on the treated (ATT) reported in DID and PSM, and the local average treatment effect (LATE) reported in RDD and IV. Our review also gathered evidence from 18 countries, four regions, – Asia, Africa, Latin America, and East Europe – various contexts, and with differences in program scale, intensity, and period, which considerably complicated study comparability and the drawing of general conclusions.<sup>15</sup> We tried to account for heterogeneity within and between studies by estimating random effects models and using moderator variables in the meta-regressions. However, the I-squared and tau-squared statistics showed a high degree of variability in the main findings.

Additional limitations of this review are worth noting. We searched for and included evidence published or made available after 2000. However, judging by other systematic reviews conducted in this field and by the publication dates of included studies, it is unlikely that more studies would be included in the review in case searches were defined with an earlier starting date.<sup>16</sup> We did not conduct a specific search in French, but we searched several databases that include studies written in other languages and we screened French language studies for inclusion in the review. We did not conduct specific searches in the RePec database, nevertheless, it is worth mentioning that we did conduct electronic searches in the Econlit database that encompasses all RePec working papers. We did not conduct moderator analysis by all types of global region, only for those regions where we had sufficient observations to undertake

> 15 Studies were done in different countries, different years, and scale as some used administrative data and other small-scale RCTs.

> 16 For instance, a paper by Grimm and Paffhausen (2015) studies a similar issue but focuses only on employment outcomes. Their search was done after 1990 and only one paper from prior to the year 2000 (Fretwell et al, 1999) was found. This paper would not qualify to enter this review as it is designed to assess active labor policy in general (not SMEs specifically) and also includes assessment of self-employment, which is not covered by this review.

28
appropriate analysis – in other words, Latin America (since the majority of the evaluated interventions were implemented in Latin America) and Africa.

Finally, we used Egger’s tests to assess whether the results discussed above reveal any indication of publication bias.<sup>17</sup> The first column in table 3 shows the results for the three primary outcomes.

Table <u>3 – Egger’s Test for Publication Bias</u>

||Firm Performance|EmploymentCreation|Labor Productivity|
|---|---|---|---|
|Slope|0.055|-0.20**|0.20**|
|(s.e.)|(0.03)|(0.08)|(0.07)|
|_p-value_|_0.109_|_0.028_|_0.027_|
|Bias|1.82|7.14*|-3.24|
|(s.e.)|(1.07)|(3.82)|(1.96)|
|_p-value_|_0.104_|_0.084_|_0.148_|

Note: Standard errors (s.e.) in parenthesis. **, * Statistically significant at 5 and 10 percent respectively.

The coefficient of the variable _bias_ is positive but only statistically significant at 11 percent (p-value = 0.104) for firm performance indicators. This result indicates weak evidence of publication bias towards studies showing positive effects of business support on SME performance indicators. The evidence of publication bias is stronger for employment creation as can be seen in the second column of table 3. The coefficient of the variable bias is positive (7.14) and statistically significant at 9 percent (p-value = 0.084). We found no evidence of publication bias for labor productivity.

# **Concluding Remarks**

This systematic review summarizes the evidence of the impact of SME-support programs that used rigorous evaluation techniques to identify the causal effect of an intervention on SME outcomes.

> 17 We used the _metabias_ command in Stata.

29
The meta-analysis found that interventions aimed at spurring SME performance had positive impacts on firm performance indicators as well as employment generation, labor productivity, exports, and investment. The sample size allowed us to look at the effect of matching grants and support to export programs through forest plots and on most individual interventions through meta-regression. Overall, the evidence shows encouraging results regarding the impact of business support on primary outcomes such as SME performance, employment creation, and labor productivity as well as on secondary outcomes such as exports, innovation, and investment.

Nevertheless, the analysis showed that region (LAC and Africa), firm size, and quality of studies (risk of bias) play an important role in the overall average effects. Importantly, despite the reasonable number of studies, few papers were classified as having low risk of bias. Consequently, the results stemming from a large number of studies with high risk of bias should be read carefully.

This review significantly contributes to understanding better the effect of various SMEsupport interventions on different outcomes while clearly setting apart SME interventions from micro-enterprise interventions, which are different in nature. Nevertheless, more work should be done to better understand what type of support works best for SMEs.

# **References**

Aivazian, V. A., & Santor, E. (2008). Financial constraints and investment: assessing the impact of a World Bank credit program on small and medium enterprises in Sri Lanka. Canadian Journal of Economics, 41(2), 475-500.

Arráiz, I., Henríquez, F., & Stucchi, R. (2013). Supplier development programs and firm performance: evidence from Chile. Small Business Economics, 41(1), 277-293.

Atkin, D., Khandelwal, A. K., & Osman, A. (2014). Exporting and Firm Performance: Evidence from a Randomized Trial. Draft: October.

Benavente, J. M., & Crespi, G. (2003). The Impact of an Associative Strategy (the PROFO Program) on Small and Medium Enterprises in Chile. SPRU Working Paper Series 88, SPRU - Science and Technology Policy Research, University of Sussex.

30
Benavente, J. M., Crespi, G., & Maffioli, A. (2007). Public Support to Firm-Level Innovation: An Evaluation of the FONTEC Program. OVE Working Papers 0507, Inter-American Development Bank, Office of Evaluation and Oversight (OVE).

Bruhn, M. (2011). License to Sell: The Effect of Business Registration Reform on Entrepreneurial Activity in Mexico. The Review of Economics and Statistics, MIT Press, 93(1), 382-386.

Bruhn, M., Karlan, D., & Schoar, A. (2012). The Impact of Consulting Services on Small and Medium Enterprises: Evidence from a Randomized Trial in Mexico, Working Papers 1010, Economic Growth Center, Yale University.

Cassano, F., & Jõeveer, K., & Svejnar,J. (2013). Cash flow vs. collateral-based credit. The Economics of Transition, The European Bank for Reconstruction and Development, 21(2), 269300.

Castillo, V., Maffioli, A., Monsalvo, A. P., Rojo, S., & Stucchi, R. (2010). Can SME Policies Improve Firm Performance? Evidence from an Impact Evaluation in Argentina. OVE Working Papers 0710, Inter-American Development Bank, Office of Evaluation and Oversight (OVE).

Castillo, V., Maffioli, A., Rojo, S., & Stucchi, R. (2011). Innovation Policy and Employment: Evidence from an Impact Evaluation in Argentina. Inter-American Development Bank Science and Technology Division, Social Sector. TECHNICAL NOTES, IDBTN-341.

Cho, Y., Honorati, M., 2014. Entrepreneurship programs around the developing world: a meta analysis. Labour Economics, 28, 110–130.

Chudnovsky, D., López, A., Rossi, M., & Ubfal, D. (2006). Evaluating A Program of Public Funding of Private Innovation Activities. An Econometric Study of FONTAR in Argentina. OVE Working Papers, Inter-American Development Bank, Office of Evaluation and Oversight (OVE) 1606.

Corseuil, L. C. H., & Moura, R. L. (2011). O Impacto do Simples Federal no Nível de Emprego da Industria Brasileira. Discussion Papers 1643, Instituto de Pesquisa Econômica Aplicada - IPEA. Crespi, G., Maffioli, A., & Melendez, M. (2011). Public Support to Innovation: the Colombian COLCIENCIAS’ Experience. Inter-American Development Bank, Social Sector Science and Technology Division, TECHNICAL NOTES No. IDB-TN-264, April.

31
De Giorgi, G., & Rahman, A. (2013). SME’s registration: Evidence from an RCT in Bangladesh. Economics Letters, Elsevier, 120(3), 573-578.

De Mel, S., McKenzie, D., & Woodruff, C. (2012). The demand for, and consequences of, formalization among informal firms in Sri Lanka. National Bureau of Economic Research (No. w18019).

De Negri, J. A., Lemos, M. B., & De Negri, F. (2006). Impact of P&D Incentive Program on the Performance and Technological Efforts of Brazilian Industrial Firms. OVE Working Papers 1406, Inter-American Development Bank, Office of Evaluation and Oversight (OVE).

Duque, J. F., & Muñoz, M. (2011). Evaluating SME Support Programs in Colombia- Chapter 4. In: Lopez-Acevedo, G., & Tan, H. W. (2011).Impact Evaluation of Small and Medium Enterprise Programs in Latin America and the Caribbean. World Bank Publications, The World Bank, 2298, March.

Fajnzylber, P., Maloney, W. F., & Montes-Rojas, G. V. (2011). Does formality improve microfirm performance? Evidence from the Brazilian SIMPLES program. Journal of Development Economics, Elsevier, 94(2), 262-276.

Gourdon, J., Marchat, J. M., Sharma, S., & Vishwanath, T. (2011). Can Matching Grants Promote Exports? Evidence from Tunisia’s FAMEX II Programme. Where to Spend the Next Million?, 81.

Grimm, M., and Paffhausen, A. (2015) Do interventions targeted at micro-entrepreneurs and small and medium-sized firms create jobs? A systematic review of the evidence for low and middle income countries, Labour Economics, 32, 67-85.

Hall, B. H., & Maffioli, A. (2008). Evaluating the Impact of Technology Development Funds in Emerging Economies: Evidence from Latin America. OVE Working Papers 0108, Inter-American Development Bank, Office of Evaluation and Oversight (OVE).

Jaramillo, M., & Diaz, J. J. (2011). Evaluating SME Support Programs in Peru- Chapter 6. In: Lopez-Acevedo, G., & Tan, H. W. (2011). Impact Evaluation of Small and Medium Enterprise Programs in Latin America and the Caribbean. World Bank Publications, The World Bank, 2298, March.

Kalume, L. R. V., Corseuil, C. H. L., & Santos, D. D. (2013). O Simples Nacional E A Formalização Das Firmas No Rio De Janeiro. Planejamento e Políticas Públicas, 40.

32
Kaplan, D. S., Piedra, E., & Seira, E. (2011). Entry regulation and business start-ups: Evidence from Mexico. Journal of Public Economics, 95(11), 1501-1515.

Karlan, D., Knight, R., & Udry, C. (2014). Consulting and Capital Experiments with Micro and Small Tailoring Enterprises in Ghana. Working Paper, October.

Leea, E. Y., & Cin, B. C. (2010). The effect of risk-sharing government subsidy on corporate R&D investment: Empirical evidence from Korea. Technological Forecasting & Social Change, 77, 881–890.

Lopez-Acevedo, G., & Tan, H. W. (2011).Impact Evaluation of Small and Medium Enterprise Programs in Latin America and the Caribbean. World Bank Publications, The World Bank, 2298, March.

Lopez-Acevedo, G., & Tan, H.W. (2005). Evaluating Training Programs for Small and Medium Enterprises Lessons from Mexico. World Bank Policy Research Working Paper 3760, November. Lopez-Acevedo, G., & Tinajero, M. (2010). Mexico: impact evaluation of SME programs using panel firm data. Policy Research Working Paper Series 5186, The World Bank.

Machado, L., Parreiras, M. A., & Peçanha, V. R. (2011). Avaliação de impacto do uso do Cartão BNDES sobre o emprego nas empresas de menor porte. Revista do BNDES 36, December.

Mano, Y., Iddrisu, A., Yoshino, Y., & Sonobe, T. (2012). How Can Micro and Small Enterprises in Sub-Saharan Africa Become More Productive? The Impacts of Experimental Basic Managerial Training. World Development, Elsevier, 40(3), 458-468.

Martincus, C. V., Carballo, J. and Garcia, P. M. (2014). Public Programmes to Promote Firm’s Exports in Developing Countries: Are There Heterogeneous Effects By Size Categories? Applied Economics, Vol. 44, 471-491.

Martincus, C. V. and Carballo, J. (2010). Beyond The Average Effects: The Distributional Impacts of Export Promotion Programs in Developing Countries. Journal of Development Economics, vol. 92, 201-214.

Martincus, C. V. and Carballo, J. (2008). Is Export Promotion Effective in Developing Countries? Firm-level Evidence on the Intensive and the Extensive Margins of Exports. Journal of International Economics, vol. 76, 89-106.

33
Martincus, C. V. and Carballo, J. (2010). Export Promotion: Bundled Service Work Better. The World Economy, 1718-1756.

McKenzie, D., & Sakho, Y. S. (2007). Does It Pay Firms to Register for Taxes? The Impact of Formality on Firm Profitability. The World Bank Development Research Group Finance and Private Sector Team, Policy Research Working Paper, 4449, December McKenzie, D., Woodruff, C., 2014. What are we learning from business training and entrepreneurship evaluations around the developing world? World Bank Research Observatory, 29 (1), 48–82.

Monteiro, J.C.M., & Assunção, J. J. (2012). Coming out of the shadows? Estimating the impact of bureaucracy simplification and tax cut on formality in Brazilian micro-enterprises. Journal of Development Economics, Elsevier, 99(1), 105-115.

Oh, I., Lee, J., Heshmati, A., & Choi, G. (2009). Evaluation of credit guarantee policy using propensity score matching. Small Business Economics, Springer, 33(3), 335-351.

Özçelik, E., & Taymaz, E. (2008). R&D support programs in developing countries: The Turkish experience. Research Policy, Elsevier, 37(2), 258-275.

Piza, C., Cravo, T., Taylor, L., Gonzalez, L., Musse, I., Furtado, I., Sierra, A. C. and Abdelnour, S. (2016). The Impacts of Business Suppport Services for Small and Medium Enterprises on Firm Performance in Low-and Middle-Income Countries: A Systematic Review. A Campbell Systematic Reviews 2016: 1.

Rand, J., & Torm, N. (2012). The Benefits of Formalization: Evidence from Vietnamese Manufacturing SMEs. World Development, Elsevier, 40(5), 983-998.

Rijkers, B., Laderchi, C. R., & Teal, F. (2010). Who benefits from promoting small enterprises? Some empirical evidence from Ethiopia. World Development, 38(4), 523–540.

Sanguinetti, P. (2005), Innovation and R&D Expenditures in Argentina: Evidence from a firm level survey. mimeo Sekkat, K. (2011). Firm Sponsored Training and Productivity in Morocco. Journal of Development Studies, 47 (9), 1391–1409.

34
Tan, H. W. (2011). Evaluating SME Support Programs in Chile- Chapter 3. In: Lopez-Acevedo, G., & Tan, H. W. (2011). Impact Evaluation of Small and Medium Enterprise Programs in Latin America and the Caribbean. World Bank Publications, The World Bank, 2298, March.

Weiss, V. V.G., López, M. S. E., & Medina, L. H. B. (2011). The impact of governmental export promotion on the firm’s export performance. Revista Latinoamericana de Administración, 47, 1- 17.

35
# **Appendix A – Search Procedures**

Electronic searches were conducted in the main international databases. Table A.1 provides the list of basic search terms used to construct complex search codes and identify studies to be included in the systematic review. Based on these terms, a detailed search strategy was set up to account for U.S. and U.K. English spelling, to seek for the most relevant studies and to restrict the search to LMICs. The search codes were developed by 3ie specialist John Eyres. The details of the search strategies are available in Piza et al. (2016) and on request. The base of the search strategy was developed using the Social Science Citation Index (ISI) and Econlit databases, two of the most important databases in economics. These codes were adapted for other databases that allow users to construct detailed codes for search terms. The databases searched are the following: 3ie database of impact evaluations, EconLit (Ovid), ABI/INFORM Global (ProQuest), PAIS International, Sociological Abst, Worldwide Political Science Abst (WPSA), ASSIA, Web of Science, that is, ‘Web of Science – Social Sciences Citation Index’, Business Source Premier (Ebsco), Academic Search Complete (Ebsco), Scopus, DAC (OECD), and Google Scholar.

# **Table A.1. Types of Intervention and Related Terms Used to Construct Complex Search Codes**

|**Types of Interventions Targeting SMEs**|**Related Search Terms**|
|---|---|
|Formalization/Business Environment<br>(Institutional Improvement)|SMEs and (formalization, business<br>environment, institutions, property<br>registration,regulatoryframeworks)|
|Exports/Access to External Markets|SMEs and (exports, certification, market<br>fairs)|
|Support for Innovation Policies|SMEs and (innovation, patents, trademarks,<br>research and development, technology<br>transfer)|
|Value Chain, Networks, and Cluster<br>Interventions|SMEs and (value chain, clusters, network,<br>localproductive systems, collective actions)|
|Training and Technical Assistance|SMEs and (training, technical assistance)|
|SME Financing/Credit Guarantee|SMEs and (finance, credit, guarantee),<br>matching grants|

36
# **Appendix B. Calculation of Effect Sizes Across Studies Using Standardized Measures**

We used two standardized measures to compare effect sizes across studies. We computed risk ratio (RR) for binary outcome variables and used standardized mean differences (SMD) for continuous variables. In most cases, the standard deviation of the whole sample (pooled standard deviation – pooled_sd) was not reported and we therefore made some assumptions to compute the SMD and its standard error (SE). For instance, in a couple of studies that reported the effects of different interventions in a long set of intermediary and final outcomes, only the difference in means and the t-statistic for the difference was recorded. When the means and standard deviation for each group were not reported we made the assumption that the standard deviation is the same in the treatment and control samples and that the covariance of the outcome variable Y between both groups is zero.<sup>18</sup> Although this assumption might be considered plausible in RCTs where the randomization is at individual level and sample sizes are similar for the treatment and control groups, it is stronger in the context of quasi-experimental studies, particularly where sample size is relatively small and numbers of observations differ sharply between treated and comparison groups. In these cases, we assumed that the standard deviation was the same, regardless of the selection process and the sample size in each group.

Whenever studies provided the sample size for the treatment and control groups at the baseline, SMD was computed using the following formulae: SMD = treatment effect/pooled_sd, for studies that used DID or matching with DID methods to compute the treatment effects.

For cases where pooled_sd is not available, we used the following:

> 18 This assumptions imply in a standard deviation (SD) of Y given by: SD(Y) = SD(beta_hat)x(2)-0.5.  See the attached file for the formulae.

37
SMD = t*[(Nt+Nc)/Nt*Nc)], where t is the t-statistic of the treatment effect coefficient in the regression model, and Nt and Nc are the numbers of treated and control observations respectively.<sup>19</sup> For studies that used small samples we corrected SMD using the following correction (see Waddington et al. 2012).<sup>20</sup> SMDcorrected = SMD*{1 – 3/[4*(Nt + Nc – 2) – 1]} We computed RR as follows (see Waddington et al., 2012): RR = [Mean(YC)+ �]/Mean(YC) for �� �0; or RR = treatment effect for �� �0.

The computation of SE of the effect sizes also requires some assumptions, particularly for RR. As discussed in Waddington et al. (2012), the SE of the error term in the regression model is the preferred option to compute RR (or SMD). In most cases, this was not available, thus we used the standard deviation of the outcome among control units at the baseline. We used the following formulae to compute SE(SMD) and SE(RR): SE(SMD) = [(Nt+Nc)/Nt*Nc) + SMD2/2*(Nt+Nc)]1/2 SE(RR) = �*{1/Nt*[Mean(YC)+ �] + 1/Nc*(Mean(YC))}, where � is the SE of the error in the regression or the standard deviation of the outcome among controls at the baseline when the former is not reported.

Finally, we made an assumption regarding sample size when this was not provided for each group separately. In cases where only the whole sample was reported, we arbitrarily split the sample equally between treated and control units.

> 19 The computation of SMD via t-test was obtained by replacing the formulae of the pooled standard deviation by a simple manipulation of the formulae of a t-test for difference in means. See Wilson (2011).

> 20 We arbitrarily defined small sample size (n) as less than 100 observations per treatment arm. According to this definition, only three studies in the final list have small samples. Most of the studies use more than 300 observations per treatment arm.

38
# **_Unit of Analysis Issues_**

Most studies use data at firm level with the great majority coming from administrative data, such as census data about formal firms or large samples of firms. The authors clustered SE accordingly in a study where the intervention took place at municipal level.

# **_Dealing with Missing Data_**

We contacted study authors to ask for missing information, such as descriptive statistics at the baseline (mean, standard deviation, and sample size and intra-cluster correlation when applicable), and received quick feedback in most cases. Unfortunately, the quality of data presented varies considerably across studies. In many cases, we were forced to make assumptions to compute SMD, RR, and the SE, for instance:

- When sample size is not provided for the treatment and control groups separately, we arbitrarily split the sample equally.

- When pooled standard deviation was not reported, we used the standard deviation of the control group to compute SE (SMD) and the t-statistic of the treatment effect coefficient to compute the SMD.

- When a study used a cluster of firms at municipality level but did not report the number of firms, we used the number of clusters (municipalities) to compute the standardized effects and SE.

- If there was no information on sample size, means, and standard deviation, the study was excluded.

- In cases where the baseline data was reported for the pooled sample of firms but estimates were provided for sub-groups of firms according to firm size, we split the sample equally among the subgroups and used the same means for subgroups as for the pooled sample.

- Some studies reported the p-values rather than the SE or t-statistics. To convert p- values into t-statistics, we used a conservative approach and used the lower value of t for cases where the coefficient was statistically significant. For instance, for cases where the p-value was between 0.051 and 0.10 we used a t-statistic of 1.65.

39
For cases where the p-value was between 0.011 and 0.05 we used a t-statistic of 1.96, and for p-values below 0.01, we used a t-statistic of 2.58.

- Where t-statistics were not available to compute SMD, we computed the pooled standard deviation using the standard deviations of the treatment and control groups and assumed a covariance between outcomes in both groups of 0.5.

- For the cases where the means at the baseline are not available or are zero, we computed RR assuming it was equal to the treatment effect coefficient.

40